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1.  Introduction  and  Project  Overview 

The  U.S.  Department  of  Defense  maintains  one  of  the  largest  healthcare  networks  in 
the  world,  supporting  in-patient  and  out-patient  care  not  just  for  the  active  military,  but 
their  families,  reserve  forces,  veterans,  and  even  civilians  local  to  various  military 
treatment  facilities  (MTF).  As  such,  each  MTF  experiences  a  wide  variety  of  patients 
and  clinical  requirements. 

Intensive  Care  Unit  (ICU)  patients  present  healthcare  teams  with  unique  challenges  and 
complex  combinations  of  life-threatening  injuries  and  illnesses.  Care  for  these  patients 
is  necessarily  multidisciplinary.  Care  providers  across  professions  must  collaborate  to 
make  effective  decisions,  develop  treatment  plans,  assess  patient  progress,  and  refine 
management  over  time.  Management  decisions,  though,  are  only  as  good  as  the 
information  available  when  they  are  made.  For  this  reason,  the  Institute  of  Medicine 
recommended  improving  access  to  accurate,  timely  information,  and  making  relevant 
information  available  at  the  point  of  patient  care  to  improve  patient  safety.  Despite 
advances  in  computer  systems  and  knowledge  resources,  communication  failures 
between  resources  and  healthcare  providers  continue  to  cause  the  majority  of 
misadventures  in  healthcare  delivery.  Critical  information  for  decision  making  remains 
difficult  to  access  and  deliver,  and  is  often  missing  at  decisive  moments. 

Healthcare  providers  in  an  ICU  environment  amount  to  a  joint  cognitive  system  that  can 
be  studied,  modeled,  and  assisted  through  scientific  methods  and  information 
technology  to  improve  decision  making  and,  thus,  improve  patient  care.  The  daily  work 
of  the  clinician  requires  knowledge  representations  as  part  of  this  joint  cognitive  system 
to  serve  as  a  map  for  the  ever-changing  environment  of  work  that  must  be  successfully 
navigated. 

As  we  envision  it,  the  Cooperative  Communication  System  (CCS)  is  part  of  a  joint 
cognitive  system  that  allows  the  healthcare  team  to  remain  connected  to  an  individual 
patient  and  to  each  other  across  time  and  space  as  the  team  delivers  patient  care.  As 
such,  it  can  keep  providers  informed  of  a  patient’s  status,  of  other  healthcare  providers’ 
activity  related  to  each  patient,  and  of  potential  discrepancies  among  healthcare 
providers’  broadly  defined,  patient  driven  goals,  specifically  defined  objectives,  and 
individually  focused  tasks.  This  type  of  networked  system  could  also  extend  beyond  the 
fixed  walls  of  a  hospital  to  incorporate  pre-hospital,  contingency  operations,  and  theater 
evacuations.  For  example,  when  a  soldier  is  injured,  a  networked  communication 
system  could  immediately  start  relaying  information  to  a  Forward  Surgical  Team  or 
Combat  Support  Hospital  to  keep  the  receiving  healthcare  team  apprised  of  the 
patient’s  status  so  that  they  can  adequately  prepare.  Handoff  on  arrival  is  then 
facilitated.  The  enhanced  communication  afforded  by  this  system  will  decrease 
complications  which  will  directly  improve  patient  outcomes. 

In  addition  to  the  improved  communication  among  providers,  this  project  explores  the 
potential  to  provide  relevant  information  to  support  clinician  decision  making.  The 
potential  exists  for  the  use  of  artificial  intelligence  algorithms  to  display  pertinent, 
prioritized  information  to  a  specific  healthcare  provider  to  support  their  current  task.  As 
more  data  becomes  available  to  the  Al  system  during  patient  care,  the  CCS  will 
continuously  (in  real  time)  improve  its  availability  and  priority  of  the  information 
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displayed.  This  type  of  decision  support  should  aid  care  providers  from  novice  to 
experienced  clinician  by  expanding  support  for  decision  making.  Through  decision 
support,  patients  might  receive  more  accurate  and  timely  diagnoses,  more  timely  and 
appropriate  testing,  and  best  evidence-based  care.  The  time  lag  from  “bench-to- 
bedside”  evidence-based  interventions  can  be  markedly  reduced.  Through  better 
communication  among  the  healthcare  team  and  by  dramatically  enhancing  the 
availability  of  salient  information  necessary  to  make  decisions,  we  expect  the  CCS  to 
reduce  complications  and  costs  and  to  improve  overall  patient  outcomes. 

The  goals  of  this  project  include: 

■  PHASE  1 :  Describe  patient  progress  through  intensive  care  to  create  a  shared 
mental  model  for  clinicians  of  all  specialties; 

■  PHASE  1 :  Provide  a  thorough  account  of  the  clinician  cognitive  work  (i.e.,  work  flow 
and  decision  requirements)  for  clinical  work  in  the  ICU,  including  accountability  of  all 
pertinent  recorded  and  non-recorded  data; 

■  PHASE  1 :  Present  design  requirements  for  the  information,  the  underlying  cognitive 
networking  rules,  and  the  display  format  of  an  IT-based  cognitive  aid  for  healthcare 
delivery  (the  Cooperative  Communication  System); 

■  PHASE  1 :  Derive  quantitative  evaluation  criteria  for  comparative  evaluation  of 
clinical  support  tools; 

■  PHASE  2:  Present  a  prototype  CCS  design  for  testing  and  implementation  in  the 
USAISR  Burn  ICU; 

■  PHASE  3:  Develop  a  test  bed  based  on  the  clinical  environment  for  Test  and 
Evaluation  of  the  CCS  and  other  clinical  support  tools. 

Phase  1  tasks  developed  a  valid  understanding  of  the  Burn  ICU  work  domain,  and 
individual  and  group  cognitive  work: 

■  Task  1.1:  Initial  Observation  of  the  Burn  ICU.  Through  observation  and  informal 
interviews,  ARA  will  identify  care  activities,  workload  requirements,  decisions  in 
patient  care,  and  the  cognitive  artifacts  clinicians  use  and  create  a  structured 
interview  guide  that  will  drive  the  remaining  work  of  this  phase. 

■  Task  1.2:  CTA  Structured  Interviews  and  Observation.  ARA  will  conduct  CTA  based 
on  the  observations  from  Task  1  and  the  interview  guide.  The  structured  interviews 
with  clinicians  will  identify  the  processes,  tools  and  cognitive  artifacts,  and  data  they 
use  during  their  patient  care  activities. 

■  Task  1.3 :  Integrated  Data  Analysis  and  Model  Development.  ARA  will  analyze  the 
data  gathered  in  Tasks  1  and  2  and  build  valid  representations  of  the  cognitive  work. 

■  Task  1.4 :  Decision  Model  and  Design  Requirements.  ARA  will  develop  specific 
decision  requirements  that  are  necessary  for  care  management  in  the  ICU. 

Phase  2  tasks  use  Phase  1’s  research  to  develop  design  requirements  for  the  IT-based 
cognitive  aid,  evaluation  criteria,  and  a  functional  prototype  of  the  CCS  design: 

■  Task  2.1 :  Scoping  and  Planning.  The  ARA  and  the  USAISR  will  translate  the  Phase 
1  findings  into  detailed  software  requirements. 
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■  Task  2.2:  Analysis.  The  ARA  and  the  USAISR  will  analyze  software  requirements 
and  write  preliminary  designs  focused  on  the  user  interfaces  and  main  architectural 
features. 

■  Task  2.3:  Design  Phase.  The  ARA,  the  USAISR,  and  the  SSCI  will  develop  the 
software  designs  including  specific  coding  and  communication  details. 

■  Task  2.4:  Implementation,  Integration  and  Testing.  ARA.  the  USAISR,  and  SSCI  will 
perform  routine  regression  testing  throughout  the  software  coding  effort.  The 
culmination  of  these  tasks  is  a  user  acceptance  test  of  the  application. 

Should  funding  be  made  available  for  Year  3,  the  project  team  will  perform  a  thorough 
evaluation  of  a  series  of  prototype  versions  with  clinicians  at  the  USAISR. 

■  Task  3.2:  Evaluation  Testing.  ARA  use  outcome-oriented  evaluation  to  assess  the 
prototype  CCS  concepts. 

2.  Accomplishments 

For  the  past  year,  the  ARA,  the  USAISR,  and  the  SSCI  project  staff  members  have 
worked  to  refine  Phase  1  results,  including  the  design  requirements,  and  to  develop  the 
CCS  prototype. 

During  Phase  2  of  the  project,  we  completed  data  collection  with  one  week-long  site 
visit  to  the  Burn  ICU  followed  by  intensive  two-day  analysis  sessions.  We  performed  this 
site  visit  to  complete  Task  1.1  and  Task  1.2  and  to  validate  the  study  results,  the 
cognitive  model  assumptions,  and  to  verify  design  requirements  (Task  1.3  and  Task 
1 .4).  Analysis  sessions  were  devoted  to  data  review  and  assessment,  and  the 
development  of  both  decision  requirements  and  representations  of  individual  and  team 
cognitive  work  performed  in  the  USAISR  Burn  ICU  (Task  1.3  and  Task  1.4).  In  addition 
to  analysis  sessions,  the  team  conducted  two  design  workshop  sessions,  where  we 
translated  findings  into  design  requirements  (Task  2.1)  including  detailed  use  cases.  We 
translated  initial  design  ideas  that  were  developed  during  the  design  sessions  into 
information  design  prototypes,  then  validated  these  with  clinicians  during  a  final  site  visit 
(Task  2.2).  The  modular  prototype  that  ARA  and  our  subcontractor  SSCI  are  developing 
is  based  on  results  of  all  preceding  tasks.  We  plan  to  test  the  first  prototype  with 
clinicians  in  December  201 4  (Task  2.3). 

a.  Data  Collection 

The  team  performed  all  on-site  observations  and  interviews,  and  removed  ail  patient 
personal  health  information  (PHI)  and  personally  identifiable  information  (Pll),  in 
accordance  with  the  IRB-prescribed  procedures. 

ARA  researchers  collected  data  at  the  USAISR  during  the  week  of  November  18-22. 
The  research  team  included  three  members  from  Cognitive  Solutions  Division, 
supported  by  a  member  of  ARA’s  San  Antonio  Office.  During  the  trip,  the  research  team 
conducted  10  interviews  with  members  of  the  USAISR  Burn  ICU  clinical  and  support 
staff  that  lasted  an  average  of  20  to  30  minutes.  The  visit  gave  the  research  team  the 
opportunity  to  fill  gaps  that  had  been  identified  during  data  analysis  sessions.  Gaps 
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included  identifying  key  areas  of  information  exchange  between  Burn  ICU  staff,  such  as 
patient  trips  to  the  OR,  shift  changes,  and  patient  transitions  into  and  out  of  the  unit. 
Each  made  a  more  focused  observation  possible  to  better  understand  the  exchange. 
The  research  team  also  collected  more  data  from  residents  and  medical  students  on 
their  roles  and  needs.  ARA  researchers  verified  findings  from  the  analysis  and  data 
synthesis,  as  well  as  beginning  system  requirements,  with  key  Burn  ICU  staff  members. 

ARA  team  members  also  circulated  through  the  BICU  to  observe  clinical  activities  and 
occasionally  ask  informal  questions  of  those  who  had  consented  to  participate  in  the 
study.  To  verify  face  validity,  the  research  team  also  reviewed  requirements  tables  from 
the  November  data  analysis  meetings  with  a  range  of  clinical  roles  (e.g.,  RN,  LVN, 
Occupational  and  Physical  Therapists,  and  Physical  Therapist  Assistant,  Respiratory 
Therapist,  and  Intensivist).  A  report  for  this  visit  is  in  Appendix  A. 

The  on-site  research  nurse  has  continued  to  assist  the  research  team  with  data 
collection.  She  helped  satisfy  IRB  human-subject  research  requirements  and,  as  of 
November  22,  2013,  had  obtained  the  consent  of  151  Burn  ICU  staff  members.  She  has 
established  a  rapport  with  Burn  ICU  staff  and  key  personnel  in  the  ISR.  She  has  also 
answered  numerous  questions  the  research  team  has  had  about  the  Burn  ICU, 
including  Burn  ICU  layout,  staffing  requirements,  staff  communication,  workflow,  staff 
duties,  and  standard  procedures. 

b.  Data  Analysis 

During  the  reporting  period,  the  ARA  team  also  conducted  a  series  of  intensive  1 -2-day 
data  analysis  sessions  on  August  27-29,  November  5-7,  and  December  16.  Elaborating 
on  the  first  year  analyses,  ARA  researchers  initially  met  to  walk  through  initial  themes  to 
be  further  explored  in  the  data.  Data  analysis  meetings  developed  both  initial 
descriptions  of  barriers  the  staff  encounter  and  cognitive  work  they  perform.  Both  are 
themes  that  appear  across  the  majority  of  data  the  team  has  collected. 

After  the  two-day  session  in  August,  a  few  of  the  team  members  met  several  times  to 
further  refine  and  define  the  themes  that  would  be  used  to  categorize  the  data.  Upon 
agreement  researchers  then  analyzed  all  of  the  interview  and  observation  notes  and 
categorized  the  data  according  to  the  agreed-upon  cognitive  themes  and  barriers  that 
Burn  ICU  personnel  experience.  The  notes  included  in  each  of  these  categories  were 
then  individually  reviewed  and  synthesized  for  further  themes  in  the  data. 

The  team  sorted  summary  statements  according  to  the  different  themes  in  order  to 
describe  barriers,  information  requirements,  and  technology  requirements.  These  were 
the  beginning  of  the  design  requirements  that  were  subsequently  confirmed  and 
expanded  upon  by  Burn  ICU  personnel.  These  findings  were  developed  into  the  Data 
Requirements  Table  is  included  in  Appendix  B.  We  refined  the  diagram  (see  Appendix 
C)  that  shows  the  information  sources  available  to  Burn  ICU  personnel,  including  who 
has  access  to  the  source,  and  what  type  of  source  it  is  (e.g.,  paper,  electronic, 
combined  paper/electronic,  and  communications).  We  also  expanded  the  working 
relationships  diagram  to  better  define  essential  roles  to  include  in  the  interface 
information  views. 
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During  the  November  data  analysis  session,  the  research  team  created  a  plan  for  the 
November  data  collection  visit  by  identifying  gaps  that  needed  to  be  filled.  The  team 
also  planned  to  verify  findings  from  the  analysis  and  the  initial  system  requirements  with 
key  Burn  ICU  personnel. 

The  December  analysis  session  was  used  to  refine  the  design  requirements  following 
the  November  data  collection  visit.  This  resulted  in  the  final  design  requirements  as  well 
as  a  complete  description  of  barriers  to  patient  care. 

The  team  delivered  the  final  report  for  Phase  1  in  February  2014. 

c.  Evaluation  Measures 

Building  on  the  design  requirements,  the  team  developed  measures  to  assess  CCS 
interface  usability,  how  well  the  system  meets  the  design  requirements,  how  well  the 
system  supports  cognitive  work.  The  research  team  has  assembled  an  initial  set  of 
metrics  (see  Appendix  D)  that  will  be  used  to  evaluate  CCS  prototype  usability. 
Subsequent  measures  now  in  development  will  be  used  to  assess  system 
requirements,  cognitive  work  support,  and  clinical  outcomes.  Evaluation  planned  for 
Year  3  will  seek  to  learn  how  it  might  affect  specific  clinical  outcomes.  We  will  continue 
to  refine  these  measures  while  we  develop  the  CCS  prototype. 

d.  Use  Cases 

The  team  developed  detailed  use  cases  (see  Appendix  E)  to  illustrate  how  the  design 
requirements  supported  the  Burn  ICU  workflow.  The  use  cases  then  served  as  the  basis 
for  machine  learning  use  cases  that  subcontractor  SSCI  developed.  ARA  worked  with 
the  ISR  and  subcontractor  SSCI,  to  develop  a  set  of  functional  requirements  and  use 
cases  (Appendix  E)  to  apply  machine  learning  capabilities  within  CCS.  We  identified 
three  functional  requirements  for  machine  learning  to  meet:  1)  Identification  of  clinician 
records  for  similar  patients,  2)  Prediction  of  future  patient  record  state,  and  3) 
Identification  of  significant  patient  and  clinician  records. 

e.  Information  Design  Prototypes 

The  ARA  research  team,  ARA  development  team  (Josh  Blomberg),  and  SSCI  (Rob 
Smith)  met  for  a  two-day  data  analysis  and  design  session  on  December  1 6-1 8,  201 3  in 
ARA’s  Fairborn,  OH  office.  The  USAISR  development  team  and  Co-PI,  Dr.  Pamplin, 
participated  by  phone  and  FaceTime.  During  this  session,  the  team  refined  and  revised 
the  design  requirements  based  on  feedback  that  was  collected  during  the  USAISR  visit. 
The  second  day  was  dedicated  to  a  design  workshop  session  in  which  the  group 
brainstormed  design  ideas  that  would  facilitate  timely,  effective,  and  efficient  patient 
care.  During  the  session,  team  members  jotted  down  ideas  and  shared  using  Post-It 
notes.  The  group  then  divided  into  smaller  teams  that  designed  rough  representations 
of  interfaces,  then  presented  them  to  the  larger  group.  The  session  provided  the 
interface  designer  with  beginning  interface  concepts  to  further  develop  and  refine.  The 
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research  team  also  updated  and  refined  the  use  cases  that  the  software  development 
team  would  need. 

The  ARA  team  also  held  a  similar  design  session  at  the  USAISR  on  February  7,  2014  to 
capture  clinician  insights.  Two  members  of  the  ARA  research  team  conducted  a  design 
workshop  including  a  discussion  of  the  use  cases  followed  by  brainstorming  activities 
and  design  workshops.  We  asked  session  participants  to  answer  questions  such  as 
“What  could  a  patient  view  look  like?”  To  respond  to  the  questions,  the  group  divided 
into  smaller  teams  according  to  clinical  roles.  Developers  divided  themselves  among  the 
different  teams.  Each  created  rough  sketches  of  a  display  that  made  sense  to  them, 
then  explained  their  concept  to  the  larger  group  and  discussed  their  reasoning  with  all 
participants. 

The  two  design  sessions  (researchers  and  developers  at  ARA  in  Fairborn  and  the 
clinicians  at  the  USAISR)  generated  nearly  the  same  amount  of  brainstorming  data  and 
paper  prototypes.  This  illustrates  the  power  of  the  team’s  methodology  and  its  ability  to 
capture  and  make  sense  of  complicated  and  complex  domains.  Based  on  these 
sessions,  the  ARA  team  then  developed  several  versions  of  the  interface  design, 
resulting  in  an  information  design  prototype  based  on  Year  1  findings  and  requirements. 

Three  members  of  the  ARA  research  team,  supported  by  one  San  Antonio  office  staff 
member,  visited  the  USAISR  in  San  Antonio  March  23-28,  2014  for  a  design  review  and 
validation  of  the  candidate  displays  by  those  who  would  use  them.  The  research  team 
identified  gaps  in  the  interface  content  and  identified  improvements  that  could  be 
completed  before  programming  began.  The  team  also  verified  the  key  systems 
requirements  with  selected  members  of  the  Burn  ICU  staff.  Using  this  information  the 
display  concepts  were  further  refined  to  create  the  information  design  prototypes  that 
are  included  in  Appendix  F. 

f.  Software  Prototype 

Josh  Blomberg,  of  ARA,  and  Jeff  Morrison,  of  SSCI,  met  with  the  Task  Area  Manager, 
Jose  Salinas,  at  the  USAISR  on  10  October  to  plan  CCS  software  development.  The 
discussion  included  USAISR  software  development  requirements  (including  SOPs),  and 
requirements  for  Information  Assurance  (IA)  and  medical  device  determination.  Their 
discussion  addressed  an  issue  that  has  posed  a  major  hurdle  for  the  project:  access  to 
relevant  medical  databases  during  development.  To  gain  access  to  actual  patient  data 
we  would  have  to  create  an  isolated  development  environment  at  the  USAISR  that  is 
not  connected  to  either  the  medical  record  or  to  the  greater  military  network.  Historical 
views  of  in-patient  data  could  then  be  placed  into  this  isolated  network  making  it 
possible  for  the  CCS  system  to  be  developed  and  tested  using  actual,  although  limited 
to  only  Essentris  in-patient  electronic  health  record  data. 

Over  the  past  year,  the  ARA  team  and  members  of  the  USAISR  staff  have  spent 
significant  time  and  effort  to  gain  access  to  relevant  medical  databases  by  pursuing 
both  internal  and  external  options.  Among  those  initiatives,  we  found  Phillips  elCU 
patient  data  closest  to  CCS  needs.  After  entering  into  a  non-disclosure  agreement  with 
Phillips,  Josh  Blomberg  of  ARA  obtained  and  loaded  the  test  Phillips  database,  then 
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analyzed  the  data  set  and  determined  that  it  would  easily  map  to  the  relational  database 
used  in  CCS. 

On  September  5th,  USAISR  Information  Management  Division  (IMD)  staff  gave 
USAISR  software  engineers  access  to  the  isolated  development  environment.  As  we 
complete  this  report,  we  are  waiting  for  necessary  electronic  medical  record  data  to  be 
loaded  into  this  environment  and  begin  to  add  necessary  software  programs  to  it. 

As  we  explored  these  data  access  options,  we  continued  to  advance  the  prototype 
development  process,  as  described  below. 

/.  Architecture 

ARA  developed  a  system  architecture  that  describes  how  to  integrate  the  CCS  user 
interface,  machine  learning,  electronic  medical  record  databases,  and  data  warehouse 
components.  We  are  pursuing  a  modular  system  architecture  connected  to  notional 
medical  databases.  During  the  past  year,  we  also  discussed  software  architectures  that 
are  approved  for  use  within  the  ISR  environment,  and  determined  that  ARA’s  proposed 
web-based  architecture  satisfies  initial  system  requirements  and  has  a  defined  pathway 
to  gain  approval  for  use  on  DHA  networks. 

ARA  has  an  initial  environment  for  software  development  and  integration,  and 
established  a  repository  for  version  control.  Accounts  have  been  created  for  ARA  and 
SSCI  personnel.  ARA  began  prototyping  a  web-based  user  interface  for  CCS  based 
upon  the  information  design  prototypes.  ARA  also  began  to  create  the  architecture  for  a 
CCS  web  site  and  for  integration  of  SSCI’s  component  technologies.  The  architecture  is 
based  on  ASP.NET  web  services  and  uses  a  Microsoft  SQL  Server  database. 

SSCI  has  also  established  a  local  repository  of  their  software  components  and  libraries 
that  are  intended  to  be  used  in  CCS.  These  include  the  PaRSA,  POINT,  and  CrossCat 
programs.  CrossCat  was  developed  as  a  part  of  DARPA’s  XDATA  program  to  find 
relationship  between  columns  of  database  content,  and  the  MedMAPP  system, 
developed  for  the  NIH  as  a  way  to  track  medically-relevant  trends  in  social  media. 

/'/'.  User  Interface 

The  CCS  user  interface  will  maintain  a  real-time  view  of  the  electronic  health  record  and 
incorporate  results  from  machine  learning  as  they  are  made  available  via  a  relational 
database.  To  that  end,  ARA  developed  a  customizable  widget-based  web  framework  for 
use  in  CCS.  Users  will  be  presented  with  a  default  CCS  user  interface  view  that  can  be 
customized.  Our  approach  allows  users  to  configure  which  data  elements  appear  on 
their  display.  Developers  will  be  able  to  note  customization  choices  that  users  make. 

Development  and  research  teams  reviewed  and  improved  versions  of  the  software 
prototype  and  reviewed  it  with  selected  USAISR  prospective  users  for  their  feedback. 
The  development  team  was  given  access  to  the  records  of  two  deceased  patients  in 
August,  as  part  of  a  IRB  sanctioned  non-human  subjects  research  protocol.  The  team  is 
now  connecting  components  of  the  interface  to  these  patient  data.  Future  versions  of 
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the  CCS  prototype  may  also  mine  data  on  how  clinicians  customize  the  interface  to 
detect  possible  relationships  between  display  customizations  and  patient  outcomes. 

iii.  Machine  Learning 

SSCI  is  responsible  for  the  development  of  the  machine  learning  features  of  CCS. 
These  will  periodically  poll  the  electronic  health  records  to  populate  a  data  warehouse. 
Their  analytic  software  will  review  those  data  to  extract  patterns  based  on  user  queries. 
The  results  will  be  transferred  to  the  CCS  database  and  used  to  invite  clinician  attention 
to  patterns  that  might  otherwise  remain  unnoticed. 

During  this  year,  SSCI  developed  a  draft  application  programming  interface  (API)  for  the 
data  analytics  module  that  will  be  delivered  during  development  and  testing  of  the  final 
CCS  prototype.  In  June,  SSCI  demonstrated  their  data  analytics  engine  based  on 
SSCI’s  implementation  of  a  scalable  Bayesian  inference  technology  known  as 
CrossCat.  The  engine  can  be  hooked  to  generic  databases,  as  well  as  during  notional 
testing  to  show  how  the  engine  can  handle  mock  medical  data.  The  engine  runs 
asynchronously  in  the  background  using  existing  patient  data.  At  the  same  time,  it  can 
also  deliver  the  specific  items  noted  below  in  real  time,  based  on  the  engine’s  current 
best  model  learned  from  the  existing  data. 

Our  inability  to  obtain  access  to  patient  data  has  so  far  prevented  SSCI  from  developing 
a  machine  learning  capability  that  is  customized  to  USAISR  data.  Instead,  SSCI  has 
developed  a  generic  engine  that  can  be  quickly  adapted  to:  a)  whatever  data  sets 
eventually  emerge,  b)  a  wide  variety  of  required  prediction  and  analysis  tasks,  as  the 
requirements  in  the  rest  of  the  project  team  clarify,  and  c)  the  scale  of  the  datasets  with 
which  we  may  have  to  handle.  The  background  engine’s  built-in  ability  to  be  distributed 
across  multiple  computers  should  enable  CCS  to  manage  large  datasets. 

Only  very  limited  Essentris  data  has  been  made  available  for  development  at  the  time  of 
this  report.  To  mitigate  this  challenge,  SSCI  has  developed  a  technique  to  expand  the 
volume  of  available  data  if  Essentris  data  access  remains  limited.  This  technique  is  to 
create  synthetic  seeded  patient  condition  data,  by  using  the  CrossCat  search  engine. 
By  inferring  the  statistics  of  relevant  data  fields  using  relevant  samples  of  data,  we  can 
use  the  engine  to  generate  synthetic  data  with  known  and  relevant  statistical  properties. 
After  this,  confirmation  of  the  system’s  correct  operation  later  in  referencing  real  patient 
data  is  straightforward. 

As  the  CCS  project  proceeds,  we  expect  adaptation  of  SSCI’s  machine  learning 
approach  should  be  a  matter  of  supervised  learning:  asking  the  search  engine  the  right 
questions,  and  providing  it  with  the  right  data.  These  adaptations  will  not  require 
significant  development  within  the  engine  itself.  After  the  initial  check-in  and 
demonstration  of  the  prototype,  we  expect  to  spend  the  remainder  of  this  project  year 
completing  adaptations  and  refinements  according  to  the  development  milestones 
(Appendix  G).  While  some  tasks  have  been  completed,  others  have  been  re-planned 
based  on  coping  with  data  availability  issues. 
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3.  Deliverables  Status 

The  deliverables  to  date  include: 

1 .  Approved  Human  Subject  Protocol:  Final  approval  completed  27  February  2013, 
Amended  protocol  approved  April  30,  2013 

2.  Visit  Reports  (x4): 

a.  First  site  visit  March  4-8,  201 3 

b.  Second  site  visit  May  20-24,  2013 

c.  Third  site  visit  July  22-25,  2013 

d.  Fourth  site  visit  November  1 8-22,  2013 

3.  Initial  Software  User  Interfaces:  January  201 4 

4.  Burn  ICU  Cognitive  Model:  February  2014 

5.  Phase  1  Final  Report:  February  2014 

6.  Finalized  User  Interfaces:  April  2014 

7.  Initial  Burn  ICU  Metrics:  September  2014 

Pending  deliverables  include: 

8.  Controlled  test  environment:  Started,  delivery  October  201 4 

9.  First  iteration  of  working  Prototype:  Started,  delivery  December  2014 

1 0.  Interim  user  evaluation  of  prototype:  expected  December  2014 

1 1 .  Final  Report  Phase  2:  expected  February  201 5 

Provision  and  delivery  will  depend  on  the  no-cost  extension  and  Year  3  funding: 

12.  Second  iteration  of  prototype:  expected  March  2015 

13.  Second  interim  user  evaluation  of  prototype:  expected  April  2015 

14.  Third  iteration  of  prototype:  expected  June  2015 

15.  Third  iteration  of  prototype:  expected  July  2015 

16.  Finalized  CCS  program:  expected  August  2015 

The  following  activities  are  planned  for  September-December  2014: 

a.  Complete  an  initial  CCS  prototype  (Task  2.4) 

b.  Finalize  assessment  metrics  and  conduct  evaluation  testing  (Task  3.2) 

c.  Develop  and  verify  initial  notions  of  scenarios  (Task  3.1  &  Task  3.3) 

d.  Conduct  usability  assessment  of  prototype  (Task  3.4) 

4.  Administrative 

Applied  Research  Associates,  Inc.  (ARA)  has  been  under  Contract  W81XWH-12-C- 
0126  to  the  U.S.  Army  Medical  Research  &  Material  Command’s  (USAMRMC) 
Telemedicine  &  Advanced  Technology  Research  Center  (TATRC)  for  two  years.  CCS 
prototype  progress  has  been  delayed  due  to  unforeseen  challenges  in  obtaining  access 
to  patient  data  and  the  databases  required  for  Phase  2  development  work.  Based  on 
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this  delay,  we  requested  and  obtained  a  no-cost  extension  to  allow  for  the  prototype  to 
be  developed  and  connected  to  a  database  with  actual  de-identified  patient  data. 

5.  Equipment  and  Supplies 

During  the  year,  the  team  acquired  a  software  development  package  with  standardized 
graphics  to  aid  the  computer  programing  team. 

6.  Reportable  Outcomes 

During  the  reporting  period,  the  research  team  has  produced  the  following  professional 
publications,  and  presentations  that  are  included  in  Appendices  H  through  L. 

Book  Chapter 

Nemeth,  C.,  Anders,  S.,  Brown,  J.,  Grome,  A.,  Crandall,  B.,  &  Pamplin,  J.  (in  press). 
Support  for  ICU  clinician  cognitive  work  through  CSE.  In  A.  Bisantz,  C.  Burns  & 

T.  Fairbanks  (Eds.),  Cognitive  engineering  applications  in  health  care.  Boca 
Raton,  FL:  Taylor  and  Francis/CRC  Press,  (editor  proof  provided) 

Proceedings 

Nemeth,  C.,  Anders,  S.,  Grome,  A.,  Crandall,  B.,  Dominguez,  C.,  Pamplin,  J.,  Mann- 
Salinas,  E.  &  Serio-Melvin,  M.  (2014,  October).  Support  for  ICU  resilience:  Using 
Cognitive  Systems  Engineering  to  build  adaptive  capacity.  Proceedings  of  the 
Systems  Man  and  Cybernetics  Society  2014  International  Symposium,  San 
Diego,  (oral  presentation) 

Presentations 

Nemeth,  C.,  Anders,  S.,  Brown,  J.,  Crandall,  B.,  Grome,  A.,  Chung,  K.,  Mann-Salinas, 
E.,  &  Pamplin,  J.  (2014,  January).  Discovery  of  Burn  ICU  critical  care 
complexities  and  the  implications  for  Health  IT  Design.  Poster  presented  at  the 
Society  of  Critical  Care  Medicine,  San  Francisco,  (poster  presentation) 

Pamplin,  J.,  Anders,  S.,  Brown,  J.,  Crandall,  B.,  Grome,  A.,  Chung,  K.,  Mann-Salinas, 
E.  &  Nemeth,  C.  (2014,  January).  Use  of  Cognitive  Systems  Engineering  to 
reveal  burn  ICU  decision-making  and  information  sources  to  aid  health 
information  technology  design  in  the  Burn  ICU.  Symposium  conducted  at  the 
Society  of  Critical  Care  Medicine,  San  Francisco,  (oral presentation) 

Nemeth,  C.,  &  Pamplin,  J.  (2014,  August).  Developing  a  cognitive  and 

communications  tool  for  Burn  ICU  clinicians.  Symposium  conducted  at  the 
Military  Health  System  Research  Symposium,  Ft.  Lauderdale,  FL.  (oral 
presentation) 

7.  Conclusions 

Cognitive  Systems  Engineering  (CSE)  group’s  study  of  work  and  information  flow 
through  this  DOD  critical  care  facility  in  Year  1  produced  a  descriptive  model  of  patient 
progress  through  the  ICU,  including  clinician  decision  requirements.  The  resulting 
description  of  the  work  domain,  information  sources,  and  clinician  work  practice  yielded 


Page  13  of  132 


W81XWH-12-C-0126 


39  requirements  for  the  CCS  prototype  that  are  now  being  developed  in  Phase  2,  and 
criteria  that  will  be  used  to  evaluate  the  prototype  in  Phase  3. 

During  Phase  2,  we  have  developed  user-oriented  use  cases  and  information  design 
prototypes  based  on  Phase  1  findings.  We  have  developed  a  programming  prototype 
that  translates  the  information  design’s  organization  and  information  into  an  interactive 
interface.  Our  subcontractor  has  also  begun  to  develop  approaches  to  machine  learning 
that  will  survey  patient  data  to  detect  patterns  and  trends  that  are  clinically  relevant.  We 
are  developing  criteria  to  evaluate  the  programming  prototype  at  the  USAISR,  which  is 
scheduled  for  early  December. 

The  main  challenge  during  this  phase  has  been  difficulty  gaining  access  to  patient  data. 
We  have  been  informed  a  development  environment  will  be  available  for  us  to  use  in 
October  2014.  We  have  requested  and  obtained  a  no-cost  extension  through  mid- 
December  201 4,  and  are  still  awaiting  Year  3  funding. 

As  the  study  continues,  the  research  team  plans  to: 

■  Finish  prototype  development,  including  the  ability  to  mine  data  for  relevant 
information. 

■  Test  and  validate  the  prototype  in  concert  with  other  IT  solutions  currently  in  use. 
Assessment  criteria  based  Year  1  research  will  be  used  to  evaluate  prototypes. 

■  Seek  avenues  to  test  and  validate  in  an  actual  clinical  setting. 

The  system  it  produces  is  expected  to  improve  communication,  information  flow,  and 
workflow  among  and  across  clinical  providers  and  support  staff. 
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9.  Appendices 


Appendix  A.  AISR  Trip  Report  -  November  2013 

23  November  2013 

From:  Christopher  Nemeth,  PhD 
To:  Betty  Levine,  TATRC 

Cc:  LtCol  Elizabeth  Mann-Salinas,  US  Army  Institute  for  Surgical  Research 

Subj:  Trip  Report:  AISR  Data  Collection  17-22  Nov  2013 

1.  Executive  Summary.  Applied  Research  Associates,  Inc.  (ARA)  is  under  Contract 
W81XWH-12-C-0126  to  the  U.S.  Army  Medical  Research  &  Material  Command’s 
(USAMRMC)  Telemedicine  &  Advanced  Technology  Research  Center  (TATRC).  The 
Cooperative  Communication  System  is  intended  to  be  part  of  a  joint  cognitive  system 
that  allows  the  healthcare  team  to  remain  connected  to  an  individual  patient  and  to  each 
other  across  time  and  space  as  the  team  delivers  patient  care.  In  addition  to  the 
improved  communication  between  providers,  this  project  explores  the  potential  to 
provide  relevant  information  to  support  clinician  decision  making.  Data  collection  visits 
during  Year  One  provides  the  descriptive  model  and  decision  requirements  for  Year 
Two  prototype  development. 

2.  Staff.  Three  members  of  the  Applied  Research  Associates  research  team  made  this 
trip  to  San  Antonio:  Shilo  Anders,  PhD,  Anna  Grome,  and  Christopher  Nemeth,  PhD. 
Dianne  Hancock  from  the  ARA  San  Antonio  office  supported  the  visit. 

3.  Activities.  All  information  was  collected  in  accordance  with  IRB-prescribed 
procedures  to  remove  patient  personal  health  information. 

a.  Interview.  To  learn  answers  to  questions  developed  in  November  data  analysis 
sessions,  conducted  10  interviews  lasting  an  average  of  20  to  30  minutes  each  with 
members  of  the  AISR  Burn  ICU  clinical  staff  in  the  following  roles: 

Intern 

Burn  OR  Chief  Nurse 
Anesthesiologist 
Bedside  Nurse  (3) 

Respiratory  Therapist 
BICU  Chief  Nurse 
Charge  Nurse  (2) 

b.  Observation.  Team  members  circulated  through  the  BICU  to  observe  clinical 
activities,  and  ask  occasional  informal  questions  of  those  who  had  consented  to 
participate  in  the  study. 
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c.  Review.  To  verify  face  validity,  Ms.  Grome,  Dr.  Nemeth,  and  Dr.  Anders  reviewed 
requirements  tables  from  the  November  data  analysis  meetings  with  a  range  of  roles: 
Bedside  Nurse  (RN,  LVN),  Occupational  and  Physical  Therapist  and  Physical  Therapist 
Assistant,  Respiratory  Therapist,  and  Intensivist. 

d.  Task  Area  Manager  Meetings. 

■  Dr.  Nemeth  and  Ms.  Grome  met  on  November  17  with  LtCol  Pamplin  to  discuss 
Phase  1 . 

■  Dr.  Nemeth  and  Dr  Anders  met  on  November  22  with  Task  Area  Manager  Dr. 
Salinas,  LtCol  Pamplin,  and  Ms.  Mario-Selvin  to  discuss  Phase  2. 

4.  Results.  The  research  team  developed  a  number  of  work  in-progress  items  to  begin 
concept  development,  support  data  analysis,  and  prepare  for  the  next  data  collection 
visit. 

a.  Interview  notes.  In-depth  notes  accounting  for  data  that  were  collected  according  to 
the  November  data  analysis  session  notes 

b.  Observation  notes.  Notes  team  members  made  during  observations  and  brief 
discussions  with  members  of  the  BICU  clinical  staff. 

c.  Annotated  requirements  tables. 

d.  Diagrams.  Scale  drawing  of  the  BICU  and  OR  spaces  floor  plan. 

e.  Annotated  draft  of  book  chapter  accepted  for  upcoming  CRC  Press  text  The 
Handbook  of  Cognitive  Engineering  in  Healthcare. 

5.  Further  work.  Next  steps  for  the  project  will  be  to: 

a.  Review  results  from  this  data  collection  visit. 

b.  Translate  field  drawing  into  finished  illustration. 

c.  Revise  requirements  tables 

6.  For  further  information,  contact  Dr.  Nemeth  at  937-825-0707,  or  cnemeth@ara.com. 
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Appendix  B.  Data  Requirements  Table 

Based  on  the  synthesis  and  integration  of  findings,  the  team  developed  an  initial  set  of  system  requirements  for  CCS 
using  the  following  questions: 

■  What  is  the  barrier  or  challenge  the  clinical  team  faces? 

■  What  does  the  clinical  team  need/require  to  overcome  that  challenge? 

■  What  system  or  display  features  could  help  address  that  challenge? 

■  What  is  the  anticipated  impact  of  meeting  that  requirement  on  team  coordination,  efficiency,  and  patient  care? 

This  Appendix  contains  the  full  set  of  initial  requirements,  the  problems  they  are  intended  to  address,  the  system  features 
suggested  by  requirements,  and  initial  ideas  about  how  system  features  might  affect  patient  care,  efficiency  and  length  of 
stay. 


Problem/Barrier 

Needs/Requirements 

System  Feature  Concepts 

Anticipated  Impacts 

No  effective  means 
to  synchronize  and 
adapt  different 
aspects  of  patient 
care  over  the 
course  of  a  shift 
(e.g.,  among  RN, 
OT/PT,  wound 
care) 

Lack  of  awareness 
around  activities/ 
events  that  are 
tightly  coupled 

No  efficient 
communication  of 
patient  status 

■  Need  to  determine  optimal 
timing  and  sequence  of 
activities 

■  Need  awareness  of 
planned/scheduled  patient 
care  activities  (e.g.,  wound 
care,  rehab,  line  changes, 
etc.) 

■  Means  to  share  the  plan 

■  Means  to  adapt  the  plan  in 
real  time  and  share 
changes  across  the  team. 

■  Bedside  Nurse  needs  to 
shift  the  goals  and  priorities 

■  Means  to  know  how 
changes  in  orders  affect/ 
change  planned  activities 

■  Means  to  know  what 

■  Visualization  of  patient  schedule 
for  shift  (patient  x  time), 
shareable  across  team 

■  Ability  to  sequence  or  overlap 
patient  care  activities 

■  Configurable  patient  groupings 

■  Prepackaged  text  to  indicate 
changes  to  schedule  (e.g., 
there’s  a  !4-hour  delay  in  PT) 

■  Sequence,  time  of  planned 
activities 

■  Provide  reason  for  delay,  and 
remedy  (using  pre-packaged 
text) 

■  Overview  through  time,  for  unit 
management 

■  Visually  connect  interdependent 
events 

■  Patients  get  needed 
care  with  fewer  delays 

■  Efficient  use  of  staff 
time 

■  Reduces  unmet 
treatment  plans  and 
intentions 

■  Supports  re-planning  - 
helps  staff  identify 
windows  of 
opportunity 
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Problem/Barrier 

Needs/Requirements 

System  Feature  Concepts 

Anticipated  Impacts 

change  across 
disciplines 

planned  events  are  and 
who  needs  to  be  there 
■  Practitioners  need  to 

understand  what’s  going  on 
with  their  group  of  patients 
across  the  shift  (whatever 
their  group  happens  to  be) 

■  Prompt/notify  appropriate  person 
when  change  impacts  their 
activity  (e.g.,  when  wound  care 
impacts  PT/OT  and  RT) 

Updated 
information  is 
available  but  not 
readily  accessible 
or  visible  to 
clinicians 
(e.g.,  cultures) 

■  Clinicians  need  to  be  aware 
that  updated  information  is 
available,  particularly  re: 
lab  cultures 

■  System  provides  news  feed  from 
lab  about  cultures. 

■  Red/am  be  r/green  about  status  of 
labs  (received  or  not;  in 
progress;  completed) 

■  Fewer  care  delays 

■  More  efficient  tracking 
and  follow  up 

■  Better  use  of  staff  time 

■  Less  reliance  on 
verbal  exchanges 

Orders  late, 
missing,  or 
overtaken/replaced 
by  other  orders 

Reliance  on  verbal 
orders  and  no 
standardized  way 
to  share  orders 

■  Need  efficient,  accurate 
way  to  specify  meds, 
procedures 

■  Physicians  need  access  to 
orders  from  Charge  Nurse’s 
checklist 

■  Physicians  need  prompts  to 
enter  orders 

■  Need  indicator  of  status  of 
order  entry  (has  it  been 
placed  or  not?) 

■  Need  indicator  of  status  of 
order  (in  process, 
completed) 

■  Physicians  need  to  be 
aware  when  entering  order 
that  it’s  the  same  as  or 
different  from  previously 

■  Order  pick  list  and  window  per 
patient  to  support  real-time  order 
entry  during  rounds 

■  Order  status  (have  orders  been 
received?  Completed?) 

■  Notify  others  if  needed  (e.g., 
infections  control) 

■  Provide  prompt  for  delayed  order 
entry  (based  on  programmable 
timing  tripwire) 

■  Display  the  information  required 
to  make  decisions  about  an 
order  available  with  the  order 
(the  relevant  parameters) 

■  Provide  molar/aggregated  view 
of  delays  for  a  given  patient 

■  System  will  track  (and  possibly 
highlight)  when  an  order  has 

■  Fewer  care  delays 

■  More  efficient  order 
entry  and  tracking 

■  Better  use  of  staff  time 
-  reduced  need  for 
repeated  follow-ups 

■  Reduced  reliance 
verbal  orders 
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Problem/Barrier 

Needs/Requirements 

System  Feature  Concepts 

Anticipated  Impacts 

entered  orders 

■  Changes  to  orders  need  to 
be  disseminated  to  wider 
team  so  that  team  has 
common  ground.  Changes 
in  orders  need  to  be 
apparent  to  whole  team 

been  changed. 

■  System  will  provide  timestamp 
for  orders 

Documentation 
requires  significant 
time  from  key 
members  of  the 
clinical  team  (RNs, 
Residents,  RTs, 
etc.)  and  is  often 
redundant 

■  Information  Management 
tools  and  processes  built 
around  efficient  use  of  staff 
time  and  effort 

■  Minimize  staff  time  required 
to  capture  information  by 
reducing  redundant 
information  gathering  and 
entry 

■  Minimize  staff  time  spent  as 
the  ‘system  integrators’ 
who  move  data  from  one 
system  to  another 

■  Need  ‘user-friendly’ 
interfaces/systems 

■  System  built  on  a  relational 
database  that  has  all  the 
information  relevant  to  a  given 
patient,  so  that  there  is  true 
interoperability:  ability  of 
separate  systems  to  cross¬ 
populate  data,  in  real  time 

■  System  supports  capturing  and 
displaying  time-based,  patient- 
based,  unit-based  data 

■  Interfaces  support  simple  data 
entry  and  pulling  information 
(faster,  more  efficient 
documentation;  errors/ 
disconnects  more  easily  spotted) 

■  System’s  ability  to  recognize 
‘repetition’  when  new 
documentation  is  introduced 
(e.g.,  ‘we  already  capture  that 
data  over  here’) 

■  System  features  that  scan  new 
documentation  requirements  for 
novel  information/redundancies 
(don’t  just  add  more) 

■  Decreased  time  spent 
entering,  moving, 
repeating,  re-entering, 
data 

■  More  time  with 
patients;  increased 
ability  to  attend  to 
patient  issues  and 
needs 

■  Decrease  cognitive 
workload 

■  Decrease  in  potential 
data  entry  errors 
(repeated  entry  of 
same  data  increases 
chance  for  error) 

Page  20  of  132 


W81XWH-12-C-0126 


Problem/Barrier 

Needs/Requirements 

System  Feature  Concepts 

Anticipated  Impacts 

Lags  in  information 
updates  means 
information  in 
system  is 
sometimes 
stale/inaccurate 

■  Means  to  indicate  if  patient 
is  highly  unstable  (because 
information  for  unstable 
patients  can  become 
inaccurate  in  short 
timeframe) 

■  Means  to  know  whether 
information  in  system  is  up- 
to-date  (e.g.,  is  this  an 
accurate  reflection  of  the 
patient’s  status  right  now?) 

■  Means  to  know  whether 
orders  are  in  process  but 
results  not  entered  into 
system  yet  (e.g.,  cultures, 
lab  results) 

■  Means  to  know  recency  of 
information  updates 

■  Means  to  capture  and 
disseminate  changes  to 
orders  that  occur  verbally 
within  sub-teams 

■  Information  should  be  time 
stamped(Q:  which  information  in 
particular?) 

■  System  should  highlight  recent 
results  (e.g.,  lab  results, 
cultures).  And  also  highlight 
orders  that  are  in  process 

■  System  should  highlight/provide 
alert  when  orders  are  changed 

■  System  should  highlight/alert 
staff  to  contraindications  (e.g., 
patient  positioning,  nutrition) 

■  Optimized  patient  care 

■  Better  use  of  staff  time 
-  reduced  need  for 
repeated  follow-ups 

■  Reduced  reliance 
verbal  orders 

■  Reduced  potential  for 
error 

Trends  are 
important 
information,  but 
can’t  get  them  from 
Essentris  or  other 

IT. 

No  ability  to  keep 
track  of  patient 
status  over  time  > 

24  hours. 

■  Clinicians  need  trend 
information 

■  Need  view  of  patient  that  is 
more  than  just  this  shift. 

Both  macro  level  view  of 
indicators  and  over  longer 
time  spans 

■  System  should  display  trend 
information  for  key  parameters 
(to  be  identified  by  clinical  staff) 

■  System  should  provide  trend 
information  over  different  time 
slices 

■  Provide  access  to  views  of 
patient  beyond  current  1 2  or  24 
hours 

■  Optimized  patient  care 

■  Increased  ability  to 
spot  changes  in 
patient  status, 
intervene  quicker 
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Problem/Barrier 

Needs/Requirements 

System  Feature  Concepts 

Anticipated  Impacts 

What  clinical  staff 
are  currently  on  the 
unit? 

■  Need  to  know  who  is 
available,  and  where  to  find 
them 

■  Need  access  to  nurse 
assignments  by  shift,  by 
patient 

■  Means  to  access 
assistance,  guidance, 
decision  makers 

■  Need  to  know  which 
specialty  is  assigned  to 
each  patient  (e.g.,  RT)  and 
patient  acuity 

■  Names  of  who  is  working  on  unit 
that  day,  with  patient 
assignments  by  room 

■  Call/staff  assignment  roster 

■  Shareable  across  disciplines 

■  Map  view  of  floor  and  display 
showing  location  of  staff. 

■  Text  paging/pre-populated 
messages 

■  ID  with  RFI  tag 

■  Allows  staff  to  readily 
know  who  is  available 
so  they  do  not  spend 
time  away  from  patient 
trying  to  locate  staff 

■  More  efficient 
communication 

■  Mitigates  care  delays 

■  Can  get  help  when  it  is 
needed 

Is  patient  ready  for 
upcoming  surgical 
procedure 

■  Need  means  to  know 
whether  patient  is  prepared 
for  procedure  (have  they 
gotten  blood  products, 
antibiotics,  consent, 
pregnancy  test) 

■  Provide  roster  of  needed  items 
(e.g.,  blood,  antibiotics)  and 
indication  of  whether  those  items 
have  been  satisfied 

■  Prevent  delay  in 
procedures 

OR  RN  does  not 
know  enough 
about  upcoming 
procedure 
to  prepare  surgical 
suite  properly 

Bedside  RN  does 
not  know  enough 
about  surgery  as  it 
is  being  performed 
to  prepare  properly 
for  patient’s  return 

■  OR  nurse  needs  procedure 
specific  description  (need 
to  know  more  about 
specific  information  needs) 

■  Bedside  Nurse  needs 
means  to  know  what  to 
expect  re  patient  needs 
following  procedure  (e.g., 
what  was  worked  on,  how 
much  blood  given  or  lost, 
sedation?) 

■  Provide  information  about 
intended  procedure 

■  Provide  information  about 
surgery  in  process  and  patient 
status 

■  Nursing  staff  better 
prepared  to  care  for 
specific  patient  needs 
at  earliest  opportunity 
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Problem/Barrier 

Needs/Requirements 

System  Feature  Concepts 

Anticipated  Impacts 

Rounding  Checklist 
not  readily 
available/access¬ 
ible  to  all  members 
of  clinical  team 

Impact  of  dropped 
tasks,  gaps,  and 
lapses  not  known 
or  tracked 

Checklist 
management  is 
unclear 

(responsibility  for 
making  sure  items 
are  completed  is 
unclear). 

■  Means  to  construct 
checklist  in  real  time 
(during  Rounds)  or 
immediately  after 

■  Means  to  post  checklist  so 
all  staff  have  ready/easy 
access 

■  Means  for  staff  to  ‘check 
off  completed  items, 
makes  notes  re:  hold  ups, 
changes/revisions 

■  Means  for  incomplete  items 
to  ‘roll  over’  to  populate 
next  day’s  check  list  and  to 
be  reviewed  at  next-day 
Rounds 

■  Checklist  needs  to  interact  with 
order  and  other  clinical  systems 

■  Unit  level  view  that  is  easy  to 
access  and  track 

■  “Roll  up”  function:  ability  to  look 
across  patients/shifts/types  of 
activities  to  examine  when  there 
are  particular  activities 
consistently  missed/delayed;  or 
care  for  a  particular  patient 
consistently  delayed 

■  System  supports  task  tripwires 
(e.g.,  timing).  Ability  to  recognize 
disconnects  between  orders  and 
implementation  (e.g.,  order 
entered,  but  not  reviewed) 

■  Provides  alerting  function  when 
tripwire  is  crossed 

■  Tripwires  are  definable  by  the 
staff 

■  Fewer  care  delays 

■  More  efficient  order 
entry  and  tracking 

■  Better  use  of  staff  time 

■  Reflect  on/improve  on 
checklist  performance 

■  Potential  unintended 
consequence: 
alarm/alert  fatigue 

Reliance  on 
clinician  to 
mentally  integrate 
data 

■  Clinicians  need  a 

holistic/macro-view  of  the 
patient’s  trajectory  (e.g., 
are  they  getting  better  or 
getting  worse  over  last 

24  hrs.?) 

■  Provide  trend  data  and  key 
indicators  (e.g.,  for  each  of  the 
main  bodily  systems) 

■  Trends  on  vitals,  wound  healing, 
medication  dosing,  infections 

■  Clinician  better  able  to 
focus  on  problem 
detection,  anticipate 
need  for  changes  in 
treatment  plans, 
optimize  decision 
making  around  patient 
care 
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Appendix  C.  Burn  ICU  Representations 

The  research  team  developed  diagrams  to  illustrate  findings  from  Phases  1  and  2, 
including  information  sources  Burn  ICU  clinicians  use,  working  relationships  they 
maintain. 

Information  Sources.  During  our  interviews  and  observations,  we  sought  to  find  out  what 
information  sources  Burn  ICU  staff  members  use.  The  following  diagram  shows  each 
source,  according  to  who  uses  it  and  the  type  of  source.  Our  data  showed 
incompatibility  among  these  sources  is  one  of  the  significant  barriers  that  the  clinicians 
face.  Success  of  the  CCS  will  need  to  integrate  this  assortment  of  sources  into  a  useful 
whole. 
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Bedside  Nurse  Communication.  During  Phase  1 ,  the  research  team  gathered 
information  about  who  individuals  communicated  with  while  working  on  the  unit.  Initial 
focus  of  this  data  collection  effort  was  on  the  Bedside  Nurses,  as  they  are  closest  to  the 
patient.  The  following  figure  illustrates  the  31  relationships  the  Bedside  Nurse  needs  to 
manage. 


CRT  Nurse 
Volunteers 
Visitors/Dignitaries 
Nursing  Staff 
Psych 

Family/Friends 

Security 

Administrative  Chief  Nurse 

Radiology 

Physicians 

Bedside  nurse  (for  report) 

Respiratory  therapist 

Occupational  therapist 

Dietician 

Pharmacy 

Researchers 
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Working  Relationships.  During  Phase  2,  we  expanded  the  Bedside  Nurse  working 
relationships  diagram  to  define  the  essential  patient  care  team.  This  enabled  the 
research  team  to  identify  the  role-specific  views  and  communication  links  that  are  most 
important  to  include  in  the  first  version  of  the  CCS  interface.  We  posed  the  question 
“Who  do  you  communicate  with  to  do  your  work?”  to  8  nurses,  5  respiratory  therapists, 
2  PT/OTs,  1  Dietician  and  1  Physician.  Results  from  that  brief  survey  yielded  the 
following  diagram.  The  rectangles  represent  the  roles.  Thicker  lines  show  that  the 
communication  was  mentioned  by  both  parties. 


Social  Worker 
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Memory  Reliance.  The  following  concept  map  diagram  illustrates  how  the  ARA  research 
team  analyzed  Burn  ICU  clinician  reliance  on  memory  to  accomplish  a  task,  and  the 
implications  it  has  for  unit  performance.  The  CCS  is  intended  to  spare  clinicians  from 
this  reliance  on  memory. 
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Appendix  D.  Evaluation  Measures 

“Metrics”  can  be  defined  as  measurable  behaviors,  work  processes,  technology 
features,  clinical  outcomes,  and  aspects  of  individual  and  team  performance.  Each 
reflects  the  problems,  needs,  and  requirements  that  we  identified  during  Phase  1 
Cognitive  Task  Analysis  (CTA). 

The  research  team  is  developing  an  initial  set  of  metrics  to  guide  the  evaluation  of  the 
CCS  during  its  development  in  order  to  provide  evidence  of  the  how  well  CCS 
addresses  those  problems,  needs  and  requirements. 

“Methods”  refers  to  the  ways  that  metrics  can  be  assessed.  We  are  considering  a  range 
of  methods  that  might  be  used  to  conduct  evaluations.  They  include  observations, 
surveys,  interviews,  extraction  of  data  from  existing  sources,  and  time/distance 
measures. 

Candidate  metrics  and  measures  for  assessing  CCS  utility  and  usability  follow: 


Technology  Metrics  and  Measures 

Cognitive  Performance  Indicators  (Wiggins  &  Cox,  2010):  Methods  might  include 

expert  review,  interviews,  surveys/questionnaires 

Metrics  include: 

■  Cue  Prominence:  Systems  should  allow  users  to  rapidly  locate  key  cues  from  the 
information  presented. 

■  Direct  Comprehension:  Systems  should  allow  users  to  directly  view  key  cues 
rather  than  requiring  users  to  manually  calculate  information  to  comprehend 
these  cues. 

■  Fine  Distinctions:  Systems  should  allow  users  to  investigate  or  at  least  access 
unfiltered  data. 

■  Enabling  Anticipation:  Systems  should  provide  information  that  allows  users  to 
anticipate  the  future  states  and  functioning  of  systems. 

■  Transparency:  Systems  should  provide  access  to  the  data  that  it  uses  and  show 
how  it  arrives  at  processed  data. 

■  Historic  Information:  Systems  should  capture  and  display  historic  information  to 
help  users  more  quickly  interpret  situations  and  diagnose  problems. 

■  Adjustable  Settings:  Systems  should  allow  users  to  refine  and  adjust  settings  as 
they  learn  more  about  a  situation. 

■  Situation  Assessment:  Systems  should  help  users  form  their  own  assessment  of 

a  situation  rather  than  provide  decisions  and  recommendations. _ 
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Cognitive  Performance  Indicators  (Wiggins  &  Cox,  2010):  Methods  might 

include  expert  review,  interviews,  surveys/questionnaires 

Metrics  include: 

■  Cue  Prominence:  Systems  should  allow  users  to  rapidly  locate  key  cues  from  the 
information  presented. 

■  Direct  Comprehension:  Systems  should  allow  users  to  directly  view  key  cues 
rather  than  requiring  users  to  manually  calculate  information  to  comprehend 
these  cues. 

■  Fine  Distinctions:  Systems  should  allow  users  to  investigate  or  at  least  access 
unfiltered  data. 

■  Enabling  Anticipation:  Systems  should  provide  information  that  allows  users  to 
anticipate  the  future  states  and  functioning  of  systems. 

■  Transparency:  Systems  should  provide  access  to  the  data  that  it  uses  and  show 
how  it  arrives  at  processed  data. 

■  Historic  Information:  Systems  should  capture  and  display  historic  information  to 
help  users  more  quickly  interpret  situations  and  diagnose  problems. 

■  Adjustable  Settings:  Systems  should  allow  users  to  refine  and  adjust  settings  as 
they  learn  more  about  a  situation. 

■  Situation  Assessment:  Systems  should  help  users  form  their  own  assessment  of 
a  situation  rather  than  provide  decisions  and  recommendations. 

SUS  (System  usability  Scale;  Brooke,  1998):  employs  a  questionnaire 

methodology,  using  a  5-point  Likert  scale. 

Metrics  include: 

■  I  think  that  I  would  like  to  use  this  system  frequently. 

■  I  found  the  system  unnecessarily  complex. 

■  I  thought  the  system  was  easy  to  use. 

■  I  think  that  I  would  need  the  support  of  a  technical  person  to  be  able  to  use  this 
system. 

■  I  found  the  various  functions  in  this  system  were  well  integrated. 

■  I  thought  there  was  too  much  inconsistency  in  this  system. 

■  I  would  imagine  that  most  people  would  learn  to  use  this  system  very  quickly. 

■  I  found  the  system  very  cumbersome  to  use. 

■  I  felt  very  confident  using  the  system. 

■  I  needed  to  learn  a  lot  of  things  before  I  could  get  going  with  this  system. _ 
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Error  Analysis:  most  useful  in  the  context  of  a  scenario,  methods  might  include 

observation,  video  recording  analysis,  think  aloud  protocols. 

Metrics  include: 

■  Use  errors:  missteps  in  the  interface  when  trying  to  complete  a  task 

■  System  errors:  page  doesn’t  display 

■  Error  recognition:  does  the  user  realize  they  made  a  mistake? 

■  Error  recovery 

■  Deviation  from  optimal  path1 _ 

Nielsen’s  Heuristics  (Nielsen,  1994):  Likely  methods  include  expert  review, 

cognitive  walkthrough 

Metrics  include: 

■  Visibility  of  system  status:  System  should  always  keep  users  informed  about  what 
is  going  on,  through  appropriate  feedback  within  reasonable  time. 

■  Match  between  system  and  real  world:  System  should  speak  the  users’  language, 
with  words,  phrases  and  concepts  familiar  to  the  user,  rather  that  system-oriented 
terms.  Follow  real-world  conventions,  making  information  appear  in  a  natural  and 
logical  order. 

■  User  control  and  freedom:  Users  often  choose  system  functions  by  mistake  and 
will  need  a  clearly  marked  "emergency  exit"  to  leave  the  unwanted  state  without 
having  to  go  through  an  extended  dialogue.  Support  undo  and  redo. 

■  Consistency  and  standards:  Users  should  not  have  to  wonder  whether  different 
words,  situations,  or  actions  mean  the  same  thing.  Follow  platform  conventions. 

■  Error  prevention:  Even  better  than  good  error  messages  is  a  careful  design  which 
prevents  a  problem  from  occurring  in  the  first  place.  Either  eliminate  error-prone 
conditions  or  check  for  them  and  present  users  with  a  confirmation  option  before 
they  commit  to  the  action. 

■  Recognition  rather  than  recall:  Minimize  the  user's  memory  load  by  making 
objects,  actions,  and  options  visible.  The  user  should  not  have  to  remember 
information  from  one  part  of  the  dialogue  to  another.  Instructions  for  use  of  the 
system  should  be  visible  or  easily  retrievable  whenever  appropriate. 

■  Flexibility  and  efficiency  of  use:  Accelerators  -  unseen  by  the  novice  user  -  may 
often  speed  up  the  interaction  for  the  expert  user  such  that  the  system  can  cater 
to  both  inexperienced  and  experienced  users.  Allow  users  to  tailor  frequent 
actions. 

■  Aesthetic  and  minimalist  design:  Dialogues  should  not  contain  information  that  is 
irrelevant  or  rarely  needed.  Every  extra  unit  of  information  in  a  dialogue  competes 
with  the  relevant  units  of  information  and  diminishes  their  relative  visibility 

■  Help  users  recognize,  diagnose,  and  recover  from  errors:  Error  messages  should 
be  expressed  in  plain  language  (no  codes),  precisely  indicate  the  problem,  and 


1  This  metric  assumes  that  there  is  an  optimal  path  through  the  tech,  to  complete  the  task,  if  that  is  the 
case  then  deviations  through  that  process  could  be  noted.  This  might  not  necessarily  be  an  error,  but 
would  suggest  inefficiencies  in  the  system  design). 
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constructively  suggest  a  solution. 

■  Help  and  documentation:  Even  though  it  is  better  if  the  system  can  be  used 
without  documentation,  it  may  be  necessary  to  provide  help  and  documentation. 

Nielsen’s  Heuristics  (Nielsen,  1994):  Likely  methods  include  expert  review, 

cognitive  walkthrough 

Metrics  include: 

■  Visibility  of  system  status:  System  should  always  keep  users  informed  about  what 
is  going  on,  through  appropriate  feedback  within  reasonable  time. 

■  Match  between  system  and  real  world:  System  should  speak  the  users’  language, 
with  words,  phrases  and  concepts  familiar  to  the  user,  rather  that  system-oriented 
terms.  Follow  real-world  conventions,  making  information  appear  in  a  natural  and 
logical  order. 

■  User  control  and  freedom:  Users  often  choose  system  functions  by  mistake  and 
will  need  a  clearly  marked  "emergency  exit"  to  leave  the  unwanted  state  without 
having  to  go  through  an  extended  dialogue.  Support  undo  and  redo. 

■  Consistency  and  standards:  Users  should  not  have  to  wonder  whether  different 
words,  situations,  or  actions  mean  the  same  thing.  Follow  platform  conventions. 

■  Error  prevention:  Even  better  than  good  error  messages  is  a  careful  design  which 
prevents  a  problem  from  occurring  in  the  first  place.  Either  eliminate  error-prone 
conditions  or  check  for  them  and  present  users  with  a  confirmation  option  before 
they  commit  to  the  action. 

■  Recognition  rather  than  recall:  Minimize  the  user's  memory  load  by  making 
objects,  actions,  and  options  visible.  The  user  should  not  have  to  remember 
information  from  one  part  of  the  dialogue  to  another.  Instructions  for  use  of  the 
system  should  be  visible  or  easily  retrievable  whenever  appropriate. 

■  Flexibility  and  efficiency  of  use:  Accelerators  -  unseen  by  the  novice  user  --  may 
often  speed  up  the  interaction  for  the  expert  user  such  that  the  system  can  cater 
to  both  inexperienced  and  experienced  users.  Allow  users  to  tailor  frequent 
actions. 

■  Aesthetic  and  minimalist  design:  Dialogues  should  not  contain  information  which 
is  irrelevant  or  rarely  needed.  Every  extra  unit  of  information  in  a  dialogue 
competes  with  the  relevant  units  of  information  and  diminishes  their  relative 
visibility. 

■  Help  users  recognize,  diagnose,  and  recover  from  errors:  Error  messages  should 
be  expressed  in  plain  language  (no  codes),  precisely  indicate  the  problem,  and 
constructively  suggest  a  solution. 

■  Help  and  documentation:  Even  though  it  is  better  if  the  system  can  be  used 
without  documentation,  it  may  be  necessary  to  provide  help  and  documentation. 
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Usability  Survey  (Anders  et  al.,  2012):  employs  a  questionnaire  methodology, 
using  a  5-point  Likert  scale  and  responses  to  open-ended  questions 

Metrics  include: 

■  CCS  contains  information  that  will  be  useful  to  me. 

■  I  am  able  to  perform  routine  tasks  effectively. 

■  I  mastered  CCS’s  major  functions  in  a  reasonable  amount  of  time. 

■  Standard  medical  language  is  used  throughout  CCS. 

■  I  am  able  to  detect  when  errors  are  made. 

■  The  interface  provides  the  information  and  controls  necessary  to  accomplish 
tasks. 

■  The  way  CCS  works  makes  sense  to  me. 

■  I  would  likely  choose  not  to  use  CCS  if  given  the  choice. 

■  I  feel  the  amount  of  interaction  with  CCS  is  just  right. 

■  My  existing  knowledge  and  skills  aids  me  in  operating  CCS. 

■  CCS  is  easy  to  understand. 

■  The  presentation  of  information  is  easy  to  read. 

■  The  organization  of  the  information  in  CCS  makes  sense. 

■  Most  of  the  people  I  work  with  will  be  able  to  use  CCS  the  first  time  they  try  it. 

■  CCS  enables  me  to  respond  effectively  to  order  requests. 

■  I  can  tell  what  CCS  is  doing  at  all  times. 

■  I  am  frustrated  when  using  CCS. 

■  CCS  effectively  alerts  me  to  adverse  patient  conditions. 

■  CCS  enables  me  to  work  quickly  when  I  am  under  time  pressure. 

■  I  will  recommend  that  other  people  in  my  unit  use  CCS. 

■  CCS  prompts  me  in  a  way  that  I  would  expect. 

■  I  am  enthusiastic  about  using  CCS. 

■  I  am  able  to  quickly  recover  when  I  make  an  error. 

■  CCS  will  change  the  way  I  take  care  of  my  patients. 

■  CCS  facilitates  timely  patient  care. 

■  What  are  your  overall  impressions  of  the  CCS  user  interface  design? 

■  What  3  things  do  you  like  most  about  CCS’s  design,  specifically  in  terms  of  what 
you  can  do  in  the  interface? 

■  What  3  things  do  you  like  least  about  CCS’s  design? 

■  What  would  you  want  to  change  before  you  were  required  to  use  CCS? 


The  research  team’s  final  evaluation  metrics  plan,  to  include  system  requirements, 
cognitive  work  support,  and  clinical  outcomes,  is  planned  for  completion  in  time  for  the 
first  CCS  Usability  Assessment  in  December  201 4. 
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Appendix  E.  Use  Cases 
User  Use  Cases 

Three  use  cases  illustrate  the  requirements  that  have  been  derived  from  Phase  1  data. 
The  requirements  are  shown  in  blue  below  each  use  case.  Features  that  relate  to 
requirements  are  shown  in  bold  type. 

Use  Case  1 :  Synchronize  Patient  Care 

Users:  Bedside  nurse,  wound  care,  RT’s,  OT’s,  Charge  Nurse 

At  0630,  a  Bedside  Nurse  has  started  his  preparation  for  the  day  shift  by  reviewing 
information  on  the  patient  he  is  responsible  for.  Opening  CCS,  he  can  see  a  roster  of 
patients  on  the  unit,  chooses  his  patient’s  “at-a-glance”  view  that  shows  recent  vital 
signs,  current  orders,  medications,  care  plan,  and  notes  from  the  night  shift.  He 
checks  the  patient’s  standing  care  plan  and  treatment  goals  (from  Essentris),  and 
reviews  orders  (from  CHCS)  that  are  pending  as  well  as  the  day’s  care  activities  that 
Wound  Care  team,  RT’s,  and  PT’s  have  recommended  and  what  times  they  can 
perform  them. 

Based  on  the  night  shift  note,  the  off-going  Bedside  Nurse  entered,  he  knows  the 
patient  will  not  be  ready  for  what  the  OT  had  recommended.  He  sends  a  pre-packaged 
text  that  will  affect  the  planned  OT  activity,  recommending  that  they  postpone  the 
treatment.  Using  the  timeline  in  the  patient  view,  he  slides  RT  and  Wound  Care  into 
slots  on  the  timeline  to  sequence  them.  The  OT  line  item  is  shown  in  grey,  indicating 
postponed,  and  the  nurse  sends  a  brief  CCS  text  message  to  the  OT  that  suggests 
postponing  to  the  next  day.  The  OT  and  Wound  Care  sessions  are  shown  in  black, 
indicating  accepted.  At  the  same  time,  CCS  displays  for  OT,  RT,  Wound  Care, 
resident,  and  Charge  Nurse  reflect  that  patient’s  procedure  status  or  time.  The  lead 
OT  sends  a  CCS  text  message  that  she’ll  put  the  procedure  on  the  proposed  care  plan 
for  the  following  day. 

At  10:45,  the  Bedside  Nurse  receives  a  CCS  text  message  that  the  wound  care  team  is 
delayed  because  care  for  a  previous  patient  took  longer  than  planned  and  the  team  will 
be  30  to  45  minutes  late.  The  Bedside  Nurse  opens  the  CCS  patient  timeline,  slides  the 
“wound  care”  session  to  a  later  position.  She  then  selects  some  pre-packaged  text, 
and  the  system  pushes  an  alert  to  the  appropriate  team  members:  to  the  RT  letting 
them  know  their  originally  scheduled  activity  with  that  patient  is  going  to  be  pushed 
back.  The  alert  also  notifies  the  burn  surgeon  letting  him  know  activities  are  being 
delayed... so  he  has  a  better  sense  of  when  dressing  will  be  down  for  him  to  examine 
the  patient’s  wounds. 
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This  use  case  addresses: 

Problem:  No  effective  means  to  synchronize  and  adapt  different  aspects  of  patient  care 
over  the  course  of  a  shift,  across  caregiver  team. 

Requirement:  System  shall  provide  access  to  a  plan  of  patient  care,  visible  to  all 
caregivers  responsible  for  that  patient  that  includes: 

■  Current  patient  status  and  top-level  assessment  Goals  and  priorities  for  those  goals 

■  Changes/updates  (indicating  that  plan  is  being  updated  when  one  caregiver  is 
working  on  it)  Schedule  of  activities  and  any  changes,  timeline 

■  Orders  and  their  status 

■  Identity  and  contact  information  for  patient’s  care  team 

Ideas  generated  for  system  features  to  meet  this  requirement: 

■  Visualization  of  patient  schedule  for  shift  (patient  x  time)  (shareable  across  team) 

■  The  ability  to  sequence  or  overlap  patient  care  activities. 

■  Configurable  patient  groupings 

■  Prepackaged  text  to  indicate  changes  to  schedule  (e.g.,  there  is  Vz  hour  delay  or 
change  in  this). 

■  Sequence,  time  of  activities 

■  Provide  reason  for  delay,  and  remedy  (using  pre-packaged  text) 

■  Overview  through  time,  for  unit  management. 

■  Visually  connect  interdependent  events 

■  Prompt/notify  appropriate  person  when  change  impacts  their  activity  (e.g.  when 
wound  care  impacts  PT/OT  and  RT). 


Use  Case  2:  Order  Tracking 

Users:  Resident,  Charge  Nurse,  Bedside  Nurse,  burn  surgeon 

While  team  is  rounding  on  a  particular  patient,  a  resident  selects  patient  view  and  a  set 
of  previous  orders  that  are  organized  in  clusters  populates  the  screen.  As  the 
attending  physician  describes  what  orders  to  enter  the  resident  selects  the  relevant 
order  form  screen.  For  orders  that  are  not  routine,  the  resident  enters  the  first  few 
letters  and  relevant  options  populate  the  screen  for  selection.  She  reviews  the  order 
set,  selects  the  order  option,  which  is  then  then  highlighted.  As  the  Charge  Nurse  reads 
back  the  attending’s  directions,  the  resident  verifies  selection  of  orders  and  submits. 
After  confirming  entry,  the  orders  on  her  screen  indicate  date/time  stamp,  and  also  are 
displayed  on  screens  of  the  patient’s  Bedside  Nurse,  burn  surgeon.  The  team  then 
moves  onto  next  patient.  At  the  end  of  rounds,  the  resident  realizes  that  one  of  the 
orders  had  already  been  entered  on  the  night  shift  and  mentions  it  to  the  attending.  The 
attending  says  “if  it’s  a  duplicate  then  just  cancel  it.”  The  resident  opens  the  patient’s 
roster  of  orders,  selects  the  order,  indicates  “delete,”  and  the  order  type  greys  out  and 
shows  “CANX”  and  date/time  stamp. 
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When  checking  the  orders  status  (from  Essentris)  the  resident  had  noticed  a  tripwire 
cue  on  the  display  that  results  for  a  blood  culture  taken  at  0400  were  due  back  from  the 
lab  by  0600.  She  sends  a  prepackaged  text  message  to  the  lab  to  learn  where  the 
results  are.  The  order  line  in  the  patient’s  status  page,  which  the  Bedside  Nurse  and 
burn  surgeon  can  also  see,  indicates  a  query  is  pending.  Curious  as  to  why  that  routine 
lab  is  late,  the  burn  surgeon  opens  a  more  molar  view  that  shows  all  pending  labs 
and  notices  that  test  is  delayed  for  all  BICU  patients.  Checking  by  phone  with  the  lab, 
he  finds  a  failed  lab  equipment  part  has  slowed  throughput,  and  sends  a  brief  CCS  text 
note  to  ail  residents  and  Bedside  Nurses  to  expect  a  delay  for  that  particular  test. 


1.  Problem:  Order  entry  is  often  delayed,  requiring  members  of  the  clinical  team  to 
track  down  residents  to  remind/ask  them  to  enter  orders.  Confusion  also  exists  around 
status  of  orders  (whether  in  process  or  complete),  whether  a  new  order  is  redundant 
with  an  existing  one,  or  whether  an  order  has  been  updated/changed. 

Requirement:  System  shall  support  real-time  order  entry  -  e.g.,  order  entry  during 
rounds  -  to  mitigate  delays. 

System  shall  enable  multiple  team  members  to  view,  update,  track,  and  process  orders 
from  a  simple  (handheld?)  application,  available  on  numerous  devices,  indicating 
changes/updates  and  current  status  of  each  order. 

Once  an  order  is  in  process,  the  system  shall  provide  team  members  who  act  on  it  with 
a  simple,  accessible  means  for  annotating  their  action  in  the  system;  the  system  shall 
update  immediately  and  push  notifications  to  subscribers 

The  system  shall  enable  team  members  to  subscribe  to  push  notifications  for  certain 
patients  about  status  of  in-process  orders/labs/procedures. 

Ideas  generated  for  system  features  to  meet  this  requirement: 

■  Order  pick  list  and  window  per  patient 

■  Order  status 

■  Notify  others  if  needed  (e.g.,  infections  control) 

-  Provide  prompt  for  delayed  order  entry  (based  on  programmable  timing  tripwire) 

■  Display  information  required  to  make  decision  about  order  available  with  the  order 
(the  relevant  parameters) 

■  Provide  molar/aggregated  view  of  delays  for  a  given  patient 

■  System  will  track  (and  possibly  highlight)  when  an  order  has  been  changed. 

■  System  will  provide  timestamp  for  orders 

(see  other  category  on  info  staleness  and  providing  timestamps). 

2.  Problem:  Lags  in  updating  the  information  means  that  information  in  the  system  can 
be  stale  or  inaccurate,  causing  lack  of  SA  for  highly  unstable  patients. 
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Requirement: 

The  system  shall  allow  the  clinicians  to  know/assess  whether  the  information  in  the 
system  is  current  or  out  of  date. 

Ideas  generated  for  system  features  to  meet  this  requirement: 

■  Information  should  be  time  stamped  (Q:  which  information  in  particular?) 

■  System  should  highlight  recent  results  -  e.g.,  lab  results,  cultures.  And  also  highlight 
orders  that  are  in  process.  (Q:  What  else?) 

■  System  should  highlight/provide  alert  when  orders  are  changed. 

■  System  should  highlight/alert  staff  to  contraindications,  (e.g.,  patient  positioning, 
nutrition) 

3.  Problem:  Lab  cultures  are  lab  culture  orders  are  submitted,  but  requestors  are  not 
made  aware  of  status  (whether  order  has  been  received,  whether  it’s  pending,  whether 
it’s  growing  something),  including  when  results  are  in..,  resulting  in  delay  of  treatment 
and  other  issues. 

Requirement:  When  any  tests  are  ordered  (lab,  xray,  etc.),  the  system  shall  provide 
status  update  (e.g.,  order  received,  in  process,  completed),  and  push  results  notification 
to  requesters  and  caregivers  for  that  patient. 

Ideas  generated  for  system  features  to  meet  this  requirement: 

■  System  provides  news  feed  from  lab  about  cultures.  (Q:  are  there  others?) 


Use  Case  3:  Trends 

Users:  Resident,  intensivist,  social  worker,  chaplain,  Bedside  Nurse,  RT,  burn  surgeon 

With  a  population  of  1 1  patients,  staff  attention  has  focused  on  two  patients  on  the  unit 
who  are  exceptionally  fragile  and  in  need  of  a  great  deal  of  care.  The  resident  and 
intensivist  on  call  are  trying  to  better  understand  how  one  of  these  patients  have  been 
trending  over  the  last  couple  weeks.  This  is,  in  order  to  inform  a  decision  about 
whether  to  pursue  other  treatment  options  (e.g.,  more  aggressive  debridement)  or  to 
transition  to  palliative  care.  The  resident  pulls  up  the  patient’s  integrated  view  that 
shows  key  trend  information,  including: 

a.  Vital  signs  over  last  48  hours 

b.  Indicators  of  infection  02  saturation  n  such  as  sepsis  over  last  24-48  hours 

c.  Indicators  of  wound  healing  over  last  1  -2  weeks  (with  photographic  images  in 
the  specified  timeframe) 

After  looking  at  trends,  they  notify  the  social  worker  and  chaplain  via  CCS  text 
message  that  they  have  decided  to  talk  to  the  family  about  transitioning  the  patient  to 
palliative  care. 
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For  the  other  patient,  the  burn  surgeon,  resident,  and  Bedside  Nurse  look  at  the 
patient’s  data  together  to  evaluate  whether  he  is  ready  for  another  surgery.  In  the 
patient’s  integrated  view,  they  open  tabs  to  look  at  more  detailed  information  in  key 
categories: 

■  Lungs  (Respiration,  tidal  volume,  02  saturation,  radiology  images  possibly  indicating 
pneumonia) 

■  Cardio/Vascular  (blood  pressure,  arrhythmias,  peripheral  circulation) 

■  Hematology  (clotting  factor) 

■  Kidney  (electrolytes) 

■  Medications  (type,  dosage) 

They  agree  the  patient  is  ready  for  the  next  procedure.  The  burn  surgeon  has  the 
resident  open  the  patient  timeline  and  the  OR  timeline,  Seeing  an  opening  on  the  OR 
timeline,  he  slides  the  patient  token  onto  the  open  slot.  The  patient  timeline  shows  a 
“pending”  assignment  in  the  surgeon,  resident,  Bedside  Nurse  and  the  OR  views.  The 
surgeon  accepts  the  assignment  on  his  view  timeline,  and  the  patient’s  view  timeline 
shows  “confirmed”  in  the  surgeon,  resident,  Bedside  Nurse  and  the  OR  views. 

The  intensivist  wants  to  see  if  there  are  other  patients  with  less  severe  condition  who 
may  be  in  greater  need  of  attention.  The  CCS  unit  view  shows  an  indication  the 
tripwire  algorithm  generated  that  one  patient’s  02  levels  are  borderline.  Opening  the 
unit  view,  she  picks  02  saturation  as  a  single  variable  that  can  indicate  a  patient 
having  potential  difficulty.  One  patient’s  02  saturation  appears  to  be  borderline,  and  she 
opens  that  patient’s  individual  view,  choosing  a  12-day  view  of  02  levels.  She  notes 
that  the  patient’s  02  levels  over  the  past  week  have  been  at  or  just  below  what  she 
wants  to  see  on  that  patient.  She  sends  a  CCS  text  message  to  the  patient’s  resident, 
Bedside  Nurse,  and  RT  asking  for  a  recommendation  that  would  stabilize  the  patient’s 
02  levels.  Each  replies  within  15-20  minutes  with  their  view  and  the  intensivist  directs 
the  resident  via  a  CCS  text  message  to  enter  an  order  for  increased  02  ventilation 
rate  on  cannula. 

In  a  routine  check  of  unit-level  trends,  the  burn  surgeon  opens  a  query  in  CCS  on 
infections.  While  infections  are  well  known  for  the  two  patients  in  greatest  need,  the 
surgeon  also  sees  a  trend  of  increased  tests  ordered  for  a  particular  pathogen,  MRSA, 
that  has  been  considered  routine  and  limited  to  the  two  fragile  patients.  Shifting  the  time 
scale  for  MRSA  tests  to  the  week,  the  month,  and  two  months,  he  sees  a  significant 
increase  in  MRSA  test  frequency.  The  surgeon  sends  an  email  to  the  infection  control 
team  asking  for  the  duty  infection  control  MD  to  come  to  the  unit  and  review  the  trend. 


1.  Problem:  Caregivers  need  trend  and  macro-level  information  to  inform  SA, 
sensemaking  and  decision  making,  but  this  information  is  not  readily  available. 
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Requirement:  The  system  shall  provide  a  time-history  of  trend  information  at  selectable 
time  scales  for  key  patient  measures/parameters  (need  specifics,  which  parameters 
have  priority,  are  they  dependent  on  the  patient,  etc.). 

The  system  shall  provide  a  top-level  dashboard  of  defined  parameters  that  visually 
represents  each  patient’s  history  on  those  parameters  for  present  day,  over  the  past 
week,  over  the  past  month,  and  at  other  time  scales. 

The  system  shall  show  an  “at  a  glance”  and  role-specific  view  of  which  patients  are 
most  vulnerable  or  unstable. 

The  system  should  include  tripwire  algorithms  that  will  flag  and  notify  team  of  a  trending 
decline  or  emergent  instability  in  patient  health  or  progress. 

Ideas  generated  for  system  features  to  meet  this  requirement: 

a.  system  should  display  role-specific  trend  information  in  configurable  time  periods 
such  as  12  hours,  24  hours,  2  weeks. 

2.  Problem:  Lags  in  updating  the  information  means  that  information  in  the  system  can 
be  stale  or  inaccurate,  causing  lack  of  SA  for  highly  unstable  patients. 

Requirement: 

The  system  shall  allow  the  clinicians  to  know/assess  whether  the  information  in  the 
system  is  current  or  out  of  date. 

Ideas  generated  for  system  features  to  meet  this  requirement: 

a.  information  should  be  time  stamped  (Q:  which  info  in  particular?) 

b.  system  should  highlight  recent  results  -  e.g.,  lab  results,  cultures.  And  also 
highlight  orders  that  are  in  process.  (Q:  What  else?) 

c.  system  should  highlight/provide  alert  when  orders  are  changed. 

d.  system  should  highlight/alert  staff  to  contraindications,  (e.g.  patient 
positioning,  nutrition) 

Machine  Learning  Use  Cases 

We  start  by  defining  terms,  then  provide  three  use  cases  that  are  specific  to  what  the 
CCS  machine  learning  features  will  perform. 

Definitions 

Current  Patient:  The  patient  that  the  CCS  user  is  currently  treating. 

Condition  Point:  A  span  of  time  during  which  a  patient’s  condition  is  evaluated  as  an 
aggregate.  This  may  be  fixed-time-span-based  (a  given  day,  hour,  minute),  or 
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automatically  identified  by  CCS  data  analytics  (in  a  yet-to-be-defined  developer-level 
use  case). 

Final  Condition  Point:  The  Condition  Point  when  a  patient  exits  the  unit. 

Patient  Condition:  The  set  of  values  of  all  of  a  patient’s  relevant  data  fields,  for  a  given 
Condition  Point.  Note  that  for  any  given  patient,  these  values  may  not  be  fully 
populated.  In  addition  to  vital  signs,  laboratory  results,  I/Os,  and  current  treatments,  the 
condition  may  also  include  the  SOFA  score,  SAPS  2  score,  APACHE  III  score,  and 
POIP  features/treatments  scales.  This  may  also  include  calculated  rates  of  change  of 
variables,  and  measures  of  whether  they  are  near  to  inflection  points  (aggregated  trend 
information). 

Representative  Patient  Cohort:  A  set  of  patients  (represented  by  their  Patient 
Conditions),  from  the  historical  record,  that  are  inferred  by  CCS  data  analytics  to  be 
similar  to  the  Current  Patient  at  the  current  Condition  Point. 

Representative  Patient  Condition:  A  theoretical  (and  completely  populated)  Patient 
Condition,  constructed  to  best  represent  the  Current  Patient  at  the  current  Condition 
Point,  based  on  the  Representative  Patient  Cohort.  Note  that  free-text  relevant  data 
fields  are  populated  with  statistically  relevant  words  from  those  fields  in  records  of 
patients  in  the  Representative  Patient  Cohort. 

Subsequent  Representative  Patient  Condition:  A  theoretical  (and  completely  populated) 
Patient  Condition,  constructed  to  best  represent  the  Current  Patient,  based  on  the 
subsequent  Condition  Point  (one  per  patient)  for  all  patients  in  the  Representative 
Patient  Cohort. 

Final  Representative  Patient  Condition:  A  theoretical  (and  completely  populated)  Patient 
Condition,  constructed  to  best  represent  the  Current  Patient,  based  on  the  Final 
Condition  Points  of  all  patients  in  the  Representative  Patient  Cohort. 


Use  Cases 
Use  Case  1 : 

Identifying  possible  and  discrepant  clinician  actions  according  to  patient  current 
condition  and  predicted  trajectory: 

■  Actors:  User,  CCS 

■  The  use  case  begins  when  the  user  enters  the  rounds  review  widget. 

o  CCS  data  analytics  computes  a  Representative  Patient  Cohort  and 
Representative  Patient  Condition  for  the  Current  Patient  at  the  current 
Condition  Point 

o  CCS  displays  all  populated  values  of  the  Current  Patient’s  condition  in  the 
patient  identifier.  CCS  populates  the  patient  identifier  taking  into  account  the 
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patient’s  vital  signs,  laboratory  results,  I/Os  and  the  current  treatments. 

Values  that  are  taken  from  the  Current  Patient’s  Condition  are  displayed 
as  graphically  distinct  from  values  taken  from  the  Representative 
Patient  Condition. 

o  CCS  displays  the  patient’s  current  orders  on  the  left  hand  side  of  the  widget. 
The  recommended  orders  are  displayed  on  the  right  side  of  the  screen. 

■  CCS  categorizes  the  orders,  and  displays  orders  using  three  colors, 
one  for  each  of  the  following  categories: 

•  Orders  that  are  assigned  to  both  the  Representative  Patient 
Cohort  and  the  Current  Patient  are  marked  with  the  same  color 
in  both  columns. 

•  Orders  that  are  assigned  to  the  Representative  Patient  Cohort 
but  not  the  Current  Patient  are  highlighted  in  one  color  on  the 
right  side  of  screen. 

•  Orders  that  are  not  assigned  to  the  Representative  Patient 
Cohort  but  are  assigned  to  the  Current  Patient  are  highlighted  in 
another  color  on  the  left  side  of  the  screen. 

o  The  user  can  select  orders  from  the  current  orders  list  for  discontinuation 
(removed  from  the  patient’s  task  list)  or  modification  (added  to  the  patient’s 
task  list)  or  orders  from  the  suggested  list  (which  are  then  added  to  the  task 
list). 

■  The  use  case  ends  when  the  user  saves  the  tasks  to  the  task  list  and  exits  the 
widget. 

Sub  Use  Cases  involving  the  SSCI  developer-level  use  cases: 

■  Constructing  the  Representative  Patient  Cohort 

■  Constructing  the  Representative  Patient  Condition 

■  Building  a  list  of  recommended  orders. 

■  Categorizing  recommended  orders  in  a  list. 

■  Determining  orders  that  are  similar. 

Use  Case  2: 

Identifying  worsening  patient. 

■  Actors:  User,  CCS 

■  This  use  case  begins  when  the  user  opens  CCS  and  views  the  patient  identifier 

o  CCS  data  analytics  computes  a  Representative  Patient  Cohort, 

Representative  Patient  Condition,  Subsequent  Representative  Patient 
Condition,  and  Final  Representative  Patient  Condition 

o  The  user  determines  if  the  patient  data  might  be  moving  in  the  wrong 
direction  (e.g.,  an  inflection  point  where  vital  signs  are  worsening,  laboratory 
results  are  worsening,  or  the  pattern  of  current  data  suggest  the  patient  might 
get  worse  in  the  next  6-12  hours)  by  viewing  the  patient  identifier. 

o  CCS  displays  warnings  to  the  user  that  the  patient  may  be  getting  worse 
by  changing  the  color  of  the  patient  condition  identifier  or  making  it  flash. 
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■  This  use  case  ends  when  the  clinician  enters  the  rounds  review  widget  to  make 
changes  for  the  patient 

Sub  Use  Cases  involving  the  SSCI  developer-level  use  cases: 

■  Determining  that  a  patient  may  be  getting  worse. 

■  Creating  a  warning  that  a  patient  may  be  getting  worse. 

■  Building  a  list  of  warnings. 

Use  Case  3. 

Problem  list  summary  and  decision  support. 

■  Actors:  User,  CCS 

■  This  use  case  begins  when  the  user  views  the  Problem  List  within  the  CCS  Patient 
Summary. 

o  The  user  adds  a  new  problem  to  the  problem  list, 
o  CCS  data  analytics  computes  a  Representative  Patient  Cohort, 

Representative  Patient  Condition,  Subsequent  Representative  Patient 
Condition,  and  Final  Representative  Patient  Condition 

o  The  CCS  system  adds  entries  to  the  problem  list  with  suggested  problems 
(e.g.  using  the  clinician  notes,  the  machine  identifies  30%  TBSA  burns, 
pneumonia,  and  heart  failure;  using  the  data  the  machine  suggests  ARDS  - 
P:F  ratio  is  <  300,  PEEP  >  5,  on  the  ventilator,  CVP  <  18). 
o  CCS  displays  clinician  entered  problems  and  system  suggested  problems  in 
different  colors  on  the  problem  list. 

o  The  user  right  clicks  (or  similar  design)  on  the  problem  and  the  following 
options  appear: 

■  An  option  to  accept  or  reject  the  suggested  problem. 

■  An  option  to  mark  the  problem  resolved. 

•  If  the  user  clicks  this  option,  CCS  moves  it  the  bottom  of  the 
problem  list  under  the  heading  “resolved”  with  other  problems 
that  are  historical. 

■  Data  associated  with  this  problem. 

•  If  the  user  clicks  this  option,  CCS  displays  all  data  used  to 
suggest  or  is  associated  with  this  problem  from  the  medical 
record  (e.g.  ARDS  would  show  the  ABG,  Fi02,  ventilator 
settings,  CXR,  notes  mentioning  this  problem,  etc.) 

■  The  use  case  ends  when  the  user  stops  interacting  with  the  problem  list. 

Sub  Use  Cases  involving  the  SSCI  developer-level  use  cases: 

■  Identifying  problems  based  on  the  computed  representative  data. 

■  Building  a  list  of  problems. 

■  Building  a  list  of  medical  record  data  associated  with  a  problem. 
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Appendix  F.  Information  Design  Prototypes 


The  following  illustrations  show  the  information  design  that  evolved  from  the  Phase  1 
data,  analyses,  and  requirements.  They  include:  Patient  identifier,  Patient  System’s 
view,  Rounds  View  (including  “child-parent”  interactive  detail),  Rehabilitation  Therapist 
view,  Unit  view,  and  Family  member  view. 
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Figure  F-1.  Patient  Identifier 
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Figure  F-4.  Patient  system’s  view  -  rehabilitation. 
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Figure  F-5.  Interaction  Illustration. 
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Figure  F-6.  Rounds  View. 
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Figure  F-7.  Rounds  Review. 


Figure  F-8.  Patient  Schedule  Overview. 
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Appendix  G.  Machine  Learning  Development  Milestones 
Pending  availability  of  data  from  Essentris: 

Milestone  1  [01  Aug  14]:  Check  in  of  the  following  features  to  the  CCS  repository, 
developed  and  tested  on  SSCI’s  Synthetic  Demo  Patient  Condition  Database: 

1 .  Move  consecutive  condition  points  to  single  record. 

2.  Update  Synthetic  Demo  Patient  Condition  Data  to  make  the  number  of  condition 
points  flexible  per  record. 

3.  Update  the  test  harness  demo  to  find  representative  patient  cohorts  with  these 
changes. 

Milestone  2  [01  Aug  14]:  Publish  draft  CCS  Data  Analytics  Interface  based  on  the 
Notional  GUI  Design  for  CCS. 

Milestone  3  [29  Aug  14]:  Determine  (collaboratively  with  the  CCS  team)  the  subset  of 
data  fields  in  Essentris  that,  for  purposes  of  the  prototype,  define  a  condition  point. 

Milestone  4  [15  Oct  14]:  Check  in  the  following  features  to  the  CCS  repository, 
developed  and  tested  on  Synthetic  CCS  Patient  Condition  Database  based  on 
specifications  in  Milestone  3: 

1 .  Represent  series  of  condition  points  on  one  record  (replicating  techniques 
developed  with  Synthetic  Demo  Patient  Condition  data). 

2.  Update  test  harness  demo  to  find  representative  patient  cohorts  from  the  Synthetic 
CCS  Patient  Condition  Data. 

3.  Update  test  harness  demo  to  demonstrate  means  of  addressing  the  three  SSCI- 
developed  user-level  use  cases. 

4.  Publish  the  actual  CCS  Data  Analytics  Interface  to  ARA. 

Milestone  5  [15  Oct  14]:  Check  in  the  Database  Translator  that  builds  a  CCS  Patient 
Condition  Database  from  an  Essentris  Database. 

IfSSCI  receives  Sample  Essentris  De-ldentified  Data  (from  the  USAISR): 

Milestone  6a  [22  Oct  14]:  Create  a  full  size  Scaled-up  Synthetic  Essentris  Database 
from  the  Sample  De-ldentified  Database.  Test  the  Database  Translator  on  the  Scaled- 
up  Essentris  Database. 

If  SSCI  has  not  received  Sample  Essentris  Data: 

Milestone  6b  [14  Nov  14]:  Test  the  Database  Translator  on  the  Synthetic  Essentris 
Database  (SSCI  to  derive  based  on  the  Synthetic  CCS  Patient  Condition  Data). 
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Milestone  7  [1  Dec  14]:  Create  Synthetic  Seeded  Patient  Condition  Database,  seeded 
with  statistics  from  the  Complete  Essentris  Database  at  the  ISR  (SSCI  to  run  the 
Database  Translator  on  the  Complete  Essentris  Database  and  run  CrossCat  on  the 
resulting  CCS  Patient  Condition  Data  to  determine  statistics;  bring  statistic  back  to  SSCI 
and  create  the  Synthetic  Seeded  Patient  Condition  Data),  and  perform  test  queries  to 
find  representative  patient  cohorts  the  seeded  data. 

Milestone  8  [31  Dec  14]:  Use  the  Database  Translator  on  the  Complete  Essentris 
Database  at  the  ISR  to  build  a  CCS  Patient  Condition  Database,  demonstrate  finding 
representative  patient  cohorts  from  the  CCS  Patient  Condition  Database. 

Data  Descriptions: 

General: 

Essentris  Database:  Multiple  patients,  raw  patient  data  in  Essentris  format. 

CCS  Patient  Condition  Database:  Multiple  patients,  consecutive  patient  condition 
points,  in  CCS  format  structured  for  CrossCat. 

Specific: 

Synthetic  Demo  Patient  Condition  Database:  The  current  demo  database. 

Synthetic  CCS  Patient  Condition  Database:  Notional  data  for  all  CCS  patient 
condition  fields. 

Synthetic  Seeded  Patient  Condition  Database:  Synthetic  data  for  all  CCS  patient 
condition  fields,  created  using  the  CrossCat  statistics  seed  from  the  Complete  Essentris 
Database  at  the  ISR. 

Scaled-up  Synthetic  Essentris  Database:  Full-size  synthetic  Essentris  data  derived 
from  small,  de-identified  sample  Essentris  data. 

Synthetic  Essentris  Database:  Full-size  synthetic  Essentris  data  derived  from  the 
Synthetic  CCS  Patient  Condition  Database. 

Complete  Essentris  Database  at  the  ISR:  Full-size,  actual,  identified  patient  data. 
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Introduction 

Cognitive  systems  engineering  (CSE)  has  been  proven  to  be  useful  in  reveal¬ 
ing  key  aspects  of  operator  behavior  as  operators  pursue  goals  in  complex 
work  domains,  providing  the  foundation  for  the  development  of  solutions 
that  are  ecologically  valid.  Health  care  work  settings,  particularly  the  inten¬ 
sive  care  unit,  present  one  of  the  most  challenging  work  domains  for  a 
researcher  to  study.  Cognitive  engineering  methods  (Hollnagel  and  Woods 
1983;  Woods  and  Roth  1988;  Roth  et  al.  2002;  Militello  et  al.  2010)  can  be 
applied  to  understand  characteristics  of  complex  work  domains  such  as  the 
ICU  as  well  as  the  behavior  of  workers  including  clinicians  and  their  sup¬ 
port  staff.  The  use  of  CSE  methods  makes  it  possible  to  identify  key  traits  of 
health  care  work  settings,  such  as  decisions  clinicians  make,  obstacles  cli¬ 
nicians  face,  and  initiatives  they  take  to  overcome  these  obstacles  in  their 
efforts  to  restore  patients  to  the  best  possible  health.  CSE  methods  also  have 
the  potential  to  enable  workers  to  better  understand  their  unit's  performance 
and  more  successfully  adapt  to  unforeseen  challenges — in  other  words,  to  be 
resilient. 

This  chapter  describes  a  project  using  CSE  methods  that  is  underway  at 
a  burn  intensive  care  unit  (BICU)  in  a  major  military  medical  center.  This 
project  will  develop  an  ecologically  valid  computer-based  cognitive  artifact 
(Hutchins  2002)  that  will  support  individual  and  clinical  team  decisions  and 
communication. 


Background 

The  study  of  health  care  relies  on  the  use  of  proven  methods  by  qualified 
researchers.  This  is  because  work  at  the  sharp  (operator)  end  of  health  care 
is  (among  other  traits)  dense,  time-pressured,  and  complex.  Expert  workers 
can  find  it  difficult  to  be  objective  observers  of  their  own  activities  and  work 
settings.  Because  of  this,  studying  one's  own  system  may  yield  conclusions 
that  are  logical  but  may  also  miss  deeper  issues.  Attention  in  such  studies 
often  focuses  on  a  single  theme  while  excluding  the  many  elements  that 
interact  with  each  other  to  produce  a  collective  result — its  context. 
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For  example,  closed  claims  reviews  that  conclude  that  error  elimination  will 
remove  "error  causes"  ignore  the  complex  pressured  context  that  molded 
each  event.  It  assumes  that  a  claim  will  contain  all  of  the  information  that 
needs  to  be  known  about  an  adverse  outcome.  It  also  presumes  to  know  what 
caused  that  outcome,  that  it  was  caused  by  an  "error,"  and  that  its  cause  can 
be  "eliminated." 

Retrospective  records  review  relies  on  historical  documentation  in  order  to 
draw  conclusions  about  care  and  its  related  risks.  But  records  hold  little  of 
the  context,  speculation,  deliberation,  and  complex  trade-off  decisions  that 
typically  mold  any  significant  event. 

Voluntary  reporting  systems  have  been  touted  as  tools  to  incorporate  error 
reporting  and  analysis  into  the  culture  of  medicine  (Plews-Organ  et  al.  2004). 
However,  voluntary  reporting  fails  to  note  how  the  approach  is  vulnerable  to 
social  and  organizational  influences. 

Clinical  discussions  of  patient  safety  often  review  how  effective  a  single 
diagnostic  or  therapeutic  intervention  is  without  taking  other  factors  into 
account  that  would  affect  outcomes  in  actual  practice.  For  example,  Shojania 
et  al.  (2001)  tested  the  use  of  a  single  item  to  prevent  infections:  a  maximum 
sterile  barrier  when  placing  intravenous  catheters.  Some  clinicians  attempt 
to  make  system  analysis  easier  by  bounding  the  problem  through  selection 
and  management  of  a  single  variable.  Kyriacou  et  al.  (1999),  for  example, 
sought  to  measure  and  reduce  the  length  of  stay  in  the  emergency  depart¬ 
ment.  Some  clinicians  have  applied  methods  such  as  workload  assessment 
to  the  ED,  but  they  found  that  the  level  of  effort  that  is  required  makes  it 
difficult  to  routinely  use  it  as  a  measurement  tool  (Levin  et  al.  2006).  Others 
have  imported  measures  from  other  sectors  to  measure  a  single  aspect  of  ED 
operation.  For  example,  France  and  Levin  (2006)  used  the  notion  of  "system 
complexity"  to  determine  safe  capacity  during  care  demand  surges  but  con¬ 
ceded  that  phenomena  such  as  interruptions  need  to  be  added. 

Research  that  does  not  adequately  detect  or  understand  these  issues 
diverts  valuable  resources  into  low-yield  efforts.  Research  that  reveals  con¬ 
text  will  grasp  the  constraints  that  shape  opportunities  and  risks  in  practice, 
curb  the  influence  of  hindsight  and  outcome  bias,  and  yield  valid  solutions 
that  gain  traction  in  actual  work  settings  (Wears  and  Nemeth  2007).  A  cur¬ 
rent  intensive  care  unit  study  provides  an  illustration  of  how  the  use  of  CSE 
makes  that  possible. 


Research  Design  and  Methods 

Our  research  team  is  completing  the  first  part  of  a  three-year  study  to 
develop  a  computer-based  cognitive  aid  that  supports  cognitive  work  and 
communication.  While  it  is  still  in  its  early  stages,  it  can  serve  as  an  example 
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of  CSE's  value  in  health  care.  We  discuss  the  CSE  approach  in  this  chapter  in 
the  context  of  our  work  on  a  prior  project  that  described  quality  standards 
for  how  to  conduct  CSE  research. 


Quality 

Nemeth  et  al.  (2011)  described  the  use  of  CSE  in  a  Navy-funded  project  that 
demonstrated  how  to  use  the  CSE  approach  in  the  context  of  the  Department 
of  Defense  acquisition  process.  The  project's  results  would  be  used  by  govern¬ 
ment  staff  members  and  contractors  who  have  no  prior  CSE  training  or  expe¬ 
rience.  The  approach  needs  to  be  used  well  to  produce  useful  results.  How 
would  the  new  users  know  what  that  is?  The  team  conceived  of  "reasonable 
scientific  criteria"  as  a  way  to  guide  new  users  through  CSE  in  a  manner  that 
is  scientifically  rigorous  and  that  links  design  recommendations  directly  to 
operator  needs.  Using  steps  in  the  CSE  process,  the  team  considered  the  goals 
and  activities  at  each  stage,  case  studies  from  the  literature  that  exemplified 
each  stage,  and  ways  that  performance  and  scientific  rigor  could  be  evaluated 
at  each  stage.  In  order  to  do  that,  the  team  considered  three  questions: 

•  What  reliability/validity  criteria  are  important  and  reasonable  to 
apply  to  CTA  data? 

•  What  are  the  standards  of  practice,  and  what  needs  to  be  done  to 
meet  those  standards? 

•  How  can  a  rigorous  process  be  created  and  followed  while  also  being 
open  to  discovery  with  respect  to  process  and  outcome? 

Answers  to  these  questions  identified  a  set  of  quality  standards  for  each 
stage  of  the  CSE  process  (Table  7.1)  from  Nemeth  et  al.  (2011)  that  can  also  be 
applied  to  research  in  the  health  care  context. 

In  the  section  Research  Process,  we  describe  how  the  first  three  standards 
have  guided  our  efforts  during  the  project's  first  year.  The  standards  for 
"Application:  design"  and  "Evaluation"  will  guide  our  work  in  the  project's 
second  and  third  years. 


Research  Design 

Our  project's  goal  is  to  improve  patient  care  by  better  support  of  the  judg¬ 
ment  of  BICU  clinicians  and  teams  by  developing  a  cognitive  aid  that  assists 
in  decision  making  and  communication.  The  project's  three  phases  are 
scheduled  to  take  roughly  a  year  apiece  for  foundation  research,  cognitive 
aid  prototype  development,  and  prototype  assessment.  The  first-year  goal 
was  to  develop  a  thorough  description  of  individual  and  team  cognition  that 
will  provide  the  basis  for  cognitive  aid  prototype  development  in  the  second 
year  as  well  as  criteria  for  prototype  assessment  in  the  third  year. 
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TABLE  7.1 

Reasonable  Scientific  Criteria  for  CSE 


CSE  Step 

Standards 

1.  Preparation 

Clear  statements  of 

and  framing 

•  Issue  or  problem 

•  Framing  activities  outcome 

•  Method,  settings,  project  participant  selection  rationale 

2.  Knowledge 

Use  of  multiple  knowledge  elicitation  (KE)  methods 

elicitation 

Use  of  interview  and  observation  guides 

Purposeful  sampling  of  participants  and  settings 

Qualified  prepared  data  collectors 

Quality  control  protocols  (specified  format  to  document  data) 

Manage  the  dual  requirements  for  rigor  and  flexibility 

3.  Analysis  and 

Systematic,  purposeful,  and  documented  analysis  process 

representation 

Audit  trail  to  connect  data  elements  to  findings  to  design  elements 

Multiple  analysis  processes  and  multiple  passes  thru  the  data 

Qualified  analysis  team  members 

Validity  checks  on  findings 

Goal-driven  selection  of  qualitative  versus  quantitative  analysis 

Use  of  reliability  indices 

4.  Application: 

Iterative  design-build-evaluate  process 

design 

Subject  matter  experts  (SMEs)  for  credibility  checks 

Audit  trail  to  connect  data  elements,  to  findings,  to  design 

5.  Evaluation 

Clear  assessment  criteria 

Review  evaluation  results  systematically  and  purposefully 

Evaluation  methods  reflect  key  cognitive  components,  behaviors 

Outcomes  reflect  cognitive  and  behavioral  issues  critical  for  cognitive  work 
Verify  whether  the  design/changes  improve  performance 

■The  five  The  five  core  team  members  are  experienced  in  health  care  field  stud- 

member  core  ~ 

team  members"  jes  using  CSE  methods  and  are  located  remotely  from  the  research  site.  To 

was  changed  to  °  J 

■The  five  core  manage  this,  they  retained  a  licensed  vocational  nurse  (LVN)  at  the  site  to 

piease  check  help  with  the  administrative  aspects  of  research  team  visits.  All  data  collec¬ 
tion  and  human  subject  consent  were  carried  out  under  the  jurisdiction  of 
the  medical  center's  Institutional  Review  Board  (IRB),  which  reviewed  and 
approved  the  research  protocol.  In  advance  of  the  team's  first  trip  to  the  site, 
the  Co-PI  and  LVN  obtained  the  consent  of  health  care  team  members  work¬ 


ing  in  the  BICU  who  were  willing  to  participate  in  the  study.  Those  who 
declined  to  participate  were  excluded  from  observations  and  interviews. 


Research  Site 

The  research  site  is  a  BICU  located  in  a  new  wing  of  a  federally  funded 
450-bed  tertiary  care  military  academic  medical  center.  The  16-bed  unit  is 
widely  considered  to  be  one  of  the  best  of  its  kind  in  the  country.  Two  of 
the  ICU  beds  are  reserved  to  serve  as  a  postanesthesia  care  unit  (PACU), 
and  another  is  dedicated  to  support  the  center's  extracorporeal  membrane 
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oxygenation  (ECMO)  program.  Other  nearby  units  support  the  ICU,  includ¬ 
ing  a  step-down  unit,  dedicated  burn  operating  room,  and  an  outpatient 
clinic.  The  typical  census  averages  around  8  patients  but  has  risen  to  as  high 
as  13  during  our  study  period.  This  unit's  role  as  a  regional  tertiary  care  unit 
attracts  patients  who  have  the  most  severe  affliction  from  thermal,  chemical, 
mechanical,  or  electrical  burns.  It  treats  patients  with  burn-like  diseases  of 
the  skin  such  as  toxic  epidermal  necrolysis,  Stevens-Johnson  syndrome,  and 
the  autoimmune  disorder  pemphigus  vulgaris.  The  unit  also  treats  patients 
with  infections  or  trauma  that  causes  extensive  soft  tissue  damage  or  loss, 
such  as  necrotizing  fasciitis,  severe  degloving  injuries,  and  some  war-related 
trauma.  Patient  length  of  stay  ranges  from  days  to  more  than  12  months. 

Sample 

All  clinicians,  patients,  and  patients'  friends  and  family  members  are  poten¬ 
tial  participants  in  the  study.  By  the  end  of  the  study,  we  anticipate  that  over 
150  clinicians  will  be  included  in  the  sample.  Subjects  are  recruited  through 
word  of  mouth  in  coordination  with  the  BICU  medical  director  and  head 
nurse.  Patients  in  the  BICU  (or  their  legal  representative)  are  asked  at  the 
start  of  an  observation  period  to  complete  a  Health  Insurance  Portability  and 
Accountability  Act  release  before  observation  or  interview.  No  clinical  infor¬ 
mation  collection  or  recordings  are  made  in  the  presence  of  any  patient  who 
declines  to  complete  the  release.  Patient  medical  data  that  are  necessary  to 
clinical  decision  making  are  collected  without  protected  health  information 
and  are  used  only  as  examples  of  information  that  clinicians  need  to  do  their 
work. 

Methods 

The  study  of  human  behavior  requires  repeated  samples  to  capture  its  rich¬ 
ness,  complexity,  and  variation.  No  method  by  itself  can  account  for  this 
complexity.  As  a  result,  multiple  methods  need  to  be  used  in  order  to  ensure 
that  the  account  is  valid  and  as  accurate  as  possible.  The  research  design 
for  this  project  relies  on  multiple  methods  to  triangulate  data  collection  and 
analysis:  observation,  interviews,  and  artifact  analysis.  Comparison  of  data 
among  all  of  these  sources  minimizes  the  potential  bias  that  a  single  method 
may  induce. 

Observation 

In-person  observation  makes  it  possible  for  the  research  team  to  witness  the 
phenomena  of  patient  care  and  team  collaboration  in  situ.  Informal  probe 
questions  enable  the  researchers  to  request  background  and  clarifying  infor¬ 
mation  in  the  context  of  the  situation.  Observations  can  be  used  to  study 
the  ways  that  practitioners  perform  diagnoses  and  prepare,  launch,  monitor. 
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adjust,  and  complete  patient  care.  The  research  team  performs  observations 
at  various  times  throughout  the  day  and  evening  to  include  a  range  of  cir¬ 
cumstances  and  clinicians'  responses.  Conditions  can  range  from  quiet  rou¬ 
tine  to  rapid  changes.  These  can  happen  during  the  admission  or  discharge 
of  multiple  patients,  emergent  conditions  such  as  treating  rare  emergencies 
like  cardiac  arrest  or  burn  shock,  and  common  emergencies  such  as  treating 
postoperative  hemodynamic  instability. 

Observation  also  includes  informal  interviews  with  clinicians  as  they  work 
in  order  to  learn  the  bases  for  their  decisions  or  apparent  indecision,  moti¬ 
vations,  expectations,  and  preferences  that  observation  alone  cannot  reveal. 
Field  notes  that  researchers  make  during  observation  provide  data  for  analy¬ 
sis  to  reveal  patterns  among  and  across  clinicians.  Observations  make  it  pos¬ 
sible  to  describe  the  ways  that  individuals  and  groups  cope  with  complexity 
and  uncertainty.  Research  team  members  pay  particular  attention  to  heu¬ 
ristics  (rules  of  thumb),  and  clinicians  have  developed  their  expertise  and 
knowledge  about  individual  and  system  performance,  how  they  use  systems 
such  as  the  electronic  health  record,  mental  simulations  they  perform,  and 
how  they  assess  outcomes.  The  research  team  also  watches  for  how  the  unit 
members  resolve  discrepancies  and  conflicts,  negotiate  trade-off,  evaluate 
the  credibility  of  data  and  information  from  others  outside  of  the  unit,  and 
mentor  and  coach  junior  members. 

During  the  first  visit,  team  members  visited  the  unit  for  five  weekdays  dur¬ 
ing  the  day  shift  (0800-1600).  The  team  scheduled  regular  observations  on 
the  ICU  to  avoid  interfering  with  clinical  work.  Subsequent  visits  to  the  site 
also  covered  evening  and  night  shifts. 

Structured  Interviews 

Cognitive  task  analysis  (CTA)  interviews  are  used  to  elicit  knowledge  from 
clinicians  on  their  background  to  learn  point  of  view,  work  activities,  infor¬ 
mation  sources  on  which  they  rely,  and  reflections  on  the  challenges  they 
face  (Crandall  et  al.  2006). 


Artifact  Analysis 

Clinicians  use  cognitive  artifacts  to  capture  and  share  information  (Hutchins 
2000).  These  include  hard-copy  printouts  such  as  sign-out  sheets,  white 
marker  status  boards,  and  diagnostic  and  therapeutic  equipment  displays. 
They  also  include  personal  notes  and  related  items  that  individuals  find  help¬ 
ful,  which  are  not  part  of  the  formal  information  ecology.  The  research  team 
is  collecting  de-identified  examples  of  these  artifacts  that  are  maintained  by 
and  for  the  group,  as  well  as  artifacts  that  individuals  create  and  use  in  their 
work.  Both  formal  and  informal  artifacts  help  to  understand  the  inventory  of 
information  that  the  unit  develops  and  uses,  which  will  suggest  the  content 
and  flow  of  information  that  this  project's  prototype  will  help  to  manage. 
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Research  Process 

The  team  began  its  work  by  conducting  orientation  interviews  with  selected 
clinicians  at  the  research  site.  Quality  standards  described  in  Table  7.1  that 
Note  that  there  guided  our  work  are  shown  in  italics.  The  interviews  sought  information 
fab?ee7™  with  the  about  the  BICU  in  order  to  develop  an  interview  guide  that  would  be  used 
check  to  organize  data  collection  efforts  during  field  visits.  This  enabled  the  team 

to  develop  clear  statements  of  the  issues  and  challenges  and  the  outcome 
of  framing  activities.  Using  these,  the  team  could  create  the  rationale  for 
method,  settings,  and  selection  of  project  participants  at  the  research  site. 
Four  one-week  data  collection  visits  were  conducted  at  the  research  site  every 
other  month,  relying  on  quality  control  protocols  to  document  interviews 
and  observations,  and  cross-check  the  content  of  data  records.  Purposeful 
sampling  of  participants  and  settings  ensured  validity  and  reliability  of  the 
data  that  were  collected  during  each  visit.  Each  observation  period  lasted  one 
week  and  was  followed  by  a  refractory  period,  during  which  the  investiga¬ 
tors  reviewed  notes,  recordings,  and  artifacts.  Data  analysis  results  were  also 
used  to  revise  plans  and  interview  guides  for  later  data  collection  efforts. 


Data  Collection 

A  team  of  four  qualified,  prepared  data  collectors  traveled  to  the  site  for  the 
first  data  collection  visit.  They  conferred  with  the  Associate  PI  (located  at 
the  research  site)  on  ICU  census  and  plans  for  clinical  activity.  Using  mul¬ 
tiple  KE  methods  to  support  findings  consistency  and  comprehensiveness, 
they  conducted  CTA  interviews  to  account  for  each  role  in  the  clinical  care 
team.  They  accompanied  the  clinical  team  on  daily  rounds  each  morning, 
which  were  typically  held  outside  of  each  patient  room.  During  the  trip,  the 
team  managed  the  dual  requirements  for  rigor  and  flexibility  by  following 
interview  guides,  yet  taking  the  opportunity  to  shadow  participants  and  ask 
probe  questions  when  the  occasion  presented  itself.  The  team  collected  data 
firsthand  by  observing  the  phenomena  that  occurred  while  clinicians  pro¬ 
vided  care  in  the  ICU,  using  the  CSE  approach  to  describe  the  ICU  as  a  work 
domain  and  to  account  for  individual  and  team  cognitive  activities.  They 
also  collected  de-identified  examples  of  computer-based  and  hard-copy  arti¬ 
facts  that  the  staff  use  in  their  daily  work. 

Rounds  were  recorded  using  a  handheld  video  camera  to  capture  team 
interaction  and  artifact  use  and  were  de-identified  using  a  video-editing 
software.  Recordings  were  made  for  future  reference  on  how  team  members 
use  and  share  information,  including  reference  to  artifacts  such  as  sign-out 
sheets  and  task  lists.  When  clinicians  interacted  directly  with  the  patient, 
the  team  used  audio  recordings  to  capture  how  information  was  shared.  No 
video  was  taken  of  the  patients.  When  clinicians  had  time  available,  two  team 
members  conducted  a  CTA  interview  following  the  interview  guide  that  was 
developed  in  the  initial  six  months  of  the  project.  If  the  clinicians  were  not 
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available  during  the  scheduled  team  visit,  the  on-site  research  nurse  would 
help  to  organize  the  interview,  and  the  core  team  members  would  participate 
remotely. 


Data  Analysis 

Data  are  evaluated  using  goal-driven  selection  of  qualitative  vs.  quantita¬ 
tive  analysis  to  extract  patterns  and  themes.  The  research  team  gathers  for 
data  analysis  meetings  roughly  a  month  after  each  data  collection  visit.  The 

J  ^  J  Analysis  ses- 

team  has  experience  to  detect  and  elicit  patterns  through  a  systematic,  pur- sions  make  pos- 

f  r  .  b  }  T,  ,  sible  the  insight 

posetul,  and  documented  analysis  process.  Analysis  sessions  make  possible  into  what  matters 

1  J  1  J  1  in  the  research 

the  insight  into  what  matters  in  the  research  setting  and  why  it  matters  by  setting. edits 
performing  checks  on  findings  credibility,  consistency,  comprehensiveness, 
and  centrality. 

Team  members  prepare  by  reviewing  the  data  collected  from  the  most 
recent  visit  to  ensure  that  each  member  has  a  current  accurate  recollection. 

This  may  also  include  organizing  the  data  and  checking  to  make  sure  that 
they  are  complete  and  ready  to  be  analyzed.  Members  assemble  as  a  group  in 
2-3  day-long  sessions  over  a  week  to  discover  what  the  data  mean  by  looking 
for  central  questions,  issues,  and  themes.  For  example,  the  interview  guide 
sought  information  on  how  team  members  manage  work  flow.  Data  analy¬ 
sis  discussion  explored  observation  notes  and  interview  responses  for  items 
related  to  workflow. 

The  analysis  sessions  are  intense  sense-making  exercises  that  use  multiple 
analysis  processes  and  make  multiple  passes  through  the  data.  Qualified 
team  members  use  interview  notes,  observation  notes,  and  artifacts  to  find 
patterns  and  themes  in  the  collected  data  using  reliability  indices  such  as 
intercoder  reliability  (when  and  if  they  are  appropriate).  The  team  also  looks 
for  related  themes,  such  as  whether  there  is  evidence  among  the  data  that 
show  how  the  clinicians  identify  and  reconcile  goal  conflicts  or  resolve 
agendas  that  do  not  agree.  Team  members  suggest  themes  or  patterns  that 
seem  to  occur  in  the  data.  Others  challenge,  modify,  or  add  to  the  discus¬ 
sion  to  ensure  validity  checks  on  findings.  Team  members  create  diagrams, 
tables,  timelines,  and  storyboards  and  use  other  visualization  methods  to 
pose,  assemble,  and  reassemble  relationships  in  order  to  recognize  possible 
patterns  among  and  across  data.  During  these  free-flowing  exchanges,  new 
insights  rapidly  evolve  and  take  the  team  to  a  new  level  of  understanding. 

Keeping  track  of  the  logic  trail  during  these  sessions  can  be  a  challenge. 
Maintaining  the  logical  connection  from  data  through  analyses  matters, 
because  each  of  the  requirements  that  the  analyses  eventually  produce  must 
have  a  deliberate  link  to  the  data  from  which  they  were  derived.  To  keep 
track  of  these  relationships,  the  team  keeps  notes  that  maintain  an  audit  trail 
to  connect  data  elements  to  findings  to  design  elements.  Without  this  struc¬ 
ture,  it  is  easy  to  disregard  the  data,  producing  a  result  that  is  not  a  set  of 
findings  but  rather  a  collective  team  impression. 
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By  the  end  of  the  analysis  sessions,  the  team  has  deepened  their  under¬ 
standing  of  what  they  know  about  the  work  setting  and  what  occurs  there. 
They  also  have  a  clearer  sense  of  what  isn't  known  yet  and  needs  to  be 
included  in  the  plan  for  the  next  site  visit.  Later  in  the  year,  further  analy¬ 
sis  work  will  code  and  analyze  all  interview  and  observation  data  to  detect 
themes  and  barriers  and  produce  requirements  for  the  prototype. 


Limitations 

Modest  project  funding  made  it  necessary  to  study  one  site,  which  limits 
its  reliability.  The  research  team  was  not  available  on  the  unit  continuously 
during  the  study,  making  it  difficult  to  observe  momentary  changes  in  unit 
activity  such  as  clinician  responses  to  codes.  To  mitigate  that  limitation,  the 
research  nurse  was  available  at  the  research  site  to  collect  data  in  the  periods 
between  research  team  visits. 


Preliminary  Findings 

While  the  project  has  only  been  underway  for  a  brief  time,  the  first  data  col¬ 
lection  and  analysis  sessions  made  it  possible  to  describe  initial  findings  that 
include  unit  activity,  the  network  of  care  providers,  and  information  sources 
on  which  the  clinicians  rely.  These  elements  amount  to  an  initial  inventory 
of  the  work  setting  that  the  team  can  build  on  during  subsequent  site  visits. 

Unit  Activity 

While  many  activities  occur  on  the  unit  through  24  h.  Table  7.2  shows  the 
essential  events  that  occur  regularly  each  day.  Those  who  are  involved  in 
these  activities  and  the  information  resources  they  use  to  perform  them  start 
to  flesh  out  a  description  of  the  unit. 

Through  the  evening,  the  bedside  nurse  and  resident  both  monitor  and  occa¬ 
sionally  provide  medication  to  the  patient  assigned  to  their  care.  From  6:30  to 
8:00  a.m.,  the  residents  and  medical  students  examine  the  patients  and  prepare 
for  formal  multidisciplinary  rounds.  The  Assistant  Chief  Nurse  and  oncom¬ 
ing  bedside  nurses  hold  a  safety  huddle.  Off-going  and  oncoming  bedside 
nurses  review  their  patient's  condition  and  conduct  a  handoff.  The  ICU  Chief 
Nurse  reviews  the  unit  population  and  resource  needs,  and  the  unit  dietician 
reviews  patient  nutrition  plans.  At  8:00  a.m.,  the  general  rounds  begin  and 
can  last  up  to  two  or  more  hours  depending  on  a  number  of  factors  includ¬ 
ing  unit  census,  patients'  condition,  and  time  pressure.  From  8:00  a.m.  to  2:00 
p.m.,  patients  are  showered,  receive  care  for  their  wounds,  or  are  taken  to  the 
nearby  operating  room  procedures  such  as  tissue  debridement,  skin  grafting. 
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TABLE  7.2 

BICU  Schematic  Timeline — Weekdays 


Time 

Activity 

Participants 

Information  Resource 

0000-0645 

Patient  monitoring, 
occasional  medication 

Bedside  nurse;  resident 

Patient  monitors 

0630-0800 

Patient  exam,  rounds 
preparation 

Resident,  medical  student 

Sign-out  sheet;  patient 
health  record  (PHR), 
wound  flow, 
radiology  images; 
patient  monitors; 

bedside  nurse, 
off-going  resident 

0645-0700 

Safety  huddle 

Assistant  Chief  Nurse, 
oncoming  bedside  nurses 

Personal  notes 

0700-0800 

Bedside  report  and 
physical  assessment 

Off-going  bedside  nurse, 
oncoming  bedside  nurse 

Patient  monitors 

0700 

ICU  audit 

Assistant  Chief  Nurse 

Personal  notes 

0700-0730 

Metabolic  assessment 

Dietitian 

Excel  file;  PHR 

0800 

Patient  rounds 

Intensivist,  burn  surgeon, 
fellow,  resident,  bedside 
nurse,  charge  nurse,  medical 
student,  respiratory 
therapist,  occupational 
therapist,  social  worker, 
dietician,  psychiatrist 

PHR 

0800-1400 

Shower,  wound  care 

Bedside  nurse,  wound  care 
team:  RN  and  LVN 

Wound  flow 

0800-1400 

Medications 

Bedside  nurse 

0800-1400 

Surgeries 

Bum  surgeon,  OR  team 

Shadow  charts 

-1400 

Patient  exam 

Resident 

1200-1300 

-1500 

Lecture 

Afternoon  rounds 

Staff  physician,  surgical  and 
medical  residents,  medical 
students 

1530 

Plan  for  wound  care 
the  next  day 

Charge  nurse,  wound  care 
coordinator 

4T  assignments  sheet 

and  reconstructive  surgery.  The  remainder  of  the  day  includes  a  lecture  for 
residents/medical  students,  the  resident  examination  of  his/her  patient,  brief 
afternoon  rounds  to  review  what  has  been  completed  from  tasks  assigned 
during  morning  rounds,  and  an  informal  discussion  between  the  wound  care 
team  leader  and  the  charge  nurse  to  decide  patient  plans  for  the  next  day. 

Network 

Patients  on  this  BICU  typically  need  care  by  a  variety  of  specialists,  requiring 
exceptional  planning,  coordination,  and  ability  to  work  together.  Table  7.3 
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TABLE  7.3 

BICU  Patient  and  Patient  Care  Staff  Roles 


Patient 

Bedside 

Nurse 

Patient 

Family 

Attending 

Intensivist 

Bum 

Surgeon 

Licensed 
Social  Worker 

Head  nurse 

Occupational 

therapist 

Respiratory 

therapist 

Resident 

Medical 

student 

Clinical  nurse 
specialist 

ICU  nurse 

Staff  psychiatric 
nurse  practitioner 

Psychiatric 

nurse 

Unit  clerk 

ICU 

director 

Charge 

nurse 

Pharmacist 

depicts  many  of  the  roles  that  need  to  collaborate  to  create  and  manage  a 
feasible  plan  for  patient  care  across  multiple  shifts  through  the  week  and 
the  weekend.  The  roles  range  from  the  bedside  nurse,  who  serves  as  a  pri¬ 
mary  care  provider  and  kind  of  the  gatekeeper  for  patient  care  by  others, 
to  primary  care  physicians  such  as  the  intensivist  and  burn  surgeon,  and 
care  specialists  such  as  the  respiratory  and  occupational  therapists,  those 
who  care  for  members  of  the  health  care  team  such  as  the  psychiatric  nurse 
practitioner,  managers  who  assist  with  planning  and  oversight,  and  hospital 
employees  off  the  BICU  such  as  the  pharmacist.  In  a  unit  that  involves  as 
many  team  members  and  specialties  as  this  BICU,  it  can  help  to  focus  on  a 
single  most  important  element  of  the  work  domain.  In  this  unit,  the  bedside 
nurse  is  closest  to  the  patient  and  can  serve  as  a  focus  of  attention  for  the 
researcher  to  understand  crucial  working  relationships.  Figure  7.1  represents 
the  31  working  relationships  in  our  data  that  the  bedside  nurse  maintains 
in  daily  practice.  Among  all  of  these  roles,  the  bedside  nurse  interacts  most 
with  others  on  the  nursing  staff,  the  patients'  family  and  friends,  physicians 
(including  physicians  of  different  levels  of  training  and  of  different  special¬ 
ties),  rehabilitation/occupational  therapy  technicians,  and  the  clinical  lab 
and  blood  bank. 

Information  Resources 

Prior  work  by  researchers  including  Xiao  et  al.  (2001),  Wears  et  al.  (2007), 
Nemeth  et  al.  (2006),  and  Bisantz  et  al.  (2010)  has  described  the  role  of  cog¬ 
nitive  artifacts  (Hutchins  2000)  in  the  health  care  setting.  These  artifacts 
include  physical  items  that  are  either  personal  (e.g.,  a  sign-out  sheet  or  note 
on  a  scrap  of  paper)  or  informal  and  used  by  a  group  (e.g.,  marker  board),  as 
well  as  electronic  information  displays  that  are  local  (e.g.,  equipment  infor¬ 
mation  display)  or  distributed  (e.g.,  information  system  display;  electronic 
medical  record).  Figure  7.2  depicts  many  of  the  artifacts  that  the  staff  relies 
on  to  perform  individual  and  team  cognitive  work  each  day. 

Databases  and  interfaces  to  manage  them  include  the  PHR,  outpatient 
record,  blood  glucose  management,  laboratory  culture,  nurse  scheduling, 
and  radiology  images.  While  used  in  concert,  many  of  these  systems  are 
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Chaplain 
Case  manager 
Charge  nurse 
Housekeeping 
CNS 

Infection  control 
Ward  master 
Head  nurse 
Ward  clerk 


Bedside  nurse 


CRT  nurse 
Volunteers 
Visitors/ dignitaries 
Nursing  staff 
Psych 

Family/friends 

Security 

Administrative  chief  nurse 
Radiology 


Patient 
Computer  staff 
Lab/blood  bank 


Physicians 

Bedside  nurse  (for  report) 

Respiratory  therapist 

Occupational  therapist 

Dietician 

Pharmacy 

Researchers 


FIGURE  7.1 

Initial  representation  of  bedside  nurse  work  relationships.  (Copyright  ©  2013  Applied  Research 
Associates,  Inc.) 

actually  separate.  This  separation  requires  care  team  members  to  take  extra 
steps  and  make  temporary  hard-copy  notes  to  use  and  transfer  informa¬ 
tion  among  systems.  Other  information  resources  beyond  databases  include 
white  boards,  a  daily  wound  care  plan,  vital  signs  flow  list,  email/cell  phone 
roster,  landline  phone  roster,  resident  sign-out  sheet,  and  a  charge  nurse 
checklist.  The  strong  emphasis  on  research  at  the  project  site  has  made  it 
possible  for  clinicians  to  develop  their  own  formal  electronic  information 
sources  in  addition  to  the  hard-copy  artifacts  that  may  be  found  at  other 
health  care  locations.  The  Wound  Flow  software  program  makes  it  possible 
to  identify  the  location  and  condition  of  tissue  injury  and  skin  grafts.  An 
Excel  file  that  the  unit  dietitian  has  developed  makes  it  possible  to  accurately 
track  the  quality  and  amount  of  nutrition  that  is  crucial  for  burn  patient 
recovery.  The  Burn  Resuscitation  Decision  Support  software  enables  the  staff  The  entry 
to  accurately  manage  fluid  resuscitation  during  the  critical  48  h  following  2oo8"isr»t 
a  significant  burn  injury.  The  solution  that  this  project  creates  will  need  to  reference"^6 
bring  these  various  parts  of  this  information  ecology  (Nemeth  et  al.  2008)  missing  inform  a- 
together  in  order  to  form  a  cohesive  whole  for  the  unit  to  use.  We  expect  ences.1  ereer 
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that  using  the  cognitive  aid  will  enable  the  unit  staff  to  work  together  more 
effectively  and  efficiently  and,  as  a  result,  improve  patient  care  effectiveness 
and  outcomes. 


Cognitive  Work 


An  initial  review  of  the  data  indicates  that  individuals  and  teams  perform  a 
number  of  macrocognitive  (Crandall  et  al.  2006)  activities,  which  are  summa¬ 
rized  in  Table  7.4.  The  staff  performs  rework  through  bridging  and  work-around 
strategies  to  link  systems  that  don't  talk  to  each  other  in  an  effort  to  ensure  infor¬ 
mation  continuity.  For  example,  the  ABG  unit  is  not  connected  to  the  database 


for  the  electronic  PHR.  (See  Chapter  6  for  additional  examples,  and  a  proposed 
model,  for  tracking  ways  that  information  is  maintained  throughout  health 


care  systems.)  The  dynamic  activities  on  the  unit  require  negotiation  hourly/by 
shift/daily  among  individuals,  specialties,  and  those  who  have  different  levels 
of  expertise.  Allocation  of  resources  requires  planning  and  replanning  among  and 
across  patients  and  specialties  in  anticipation  of  the  patient  status  and  needs, 
and  how  to  meet  them  through  preparation  and  participation  in  events. 

TABLE  7.4 

Emergent  Themes  for  Cognitive  Work  of  Burn  ICU 


“Allocation 
of  resources 
requires  planning 
and  replanning 
among  and 
across  patients 
and  specialties 
in  anticipation  of 
the  patient  status 
and  needs,  and 
how  to  meet 
them  through 
preparation  and 
participation  in 
events."  -  edits 
OK? 


Theme 


Definition 


Rework 

Information 

continuity 

Negotiation 

Scheduling 

Anticipation 

Coordination 

Clarification 

Resources 

Tasking 

Cross-checking 

Tracking 

Gaps 


Bridging  and  work-around  strategies  to  link  systems  that  don't  talk  to  each 
other. 

Arterial  blood  gas  (ABG)  does/ doesn't  connect  to  electronic  PHR  An 
additional  volume  needs  to  be  created  for  a  very  long  term  care  patient 

Among  individuals  and  care  specialties,  team  member  levels  of  knowledge 
and  expertise  are  dynamic,  which  requires  negotiation  by  the  hour,  shift, 
and  day. 

Planning  and  replanning  among  and  across  specialties. 

Patient  status,  needs,  and  how  to  meet  them;  preparation  and  participation 
in  events. 

Collaboration  requires  expression  of  expectations,  prioritization, 
agreement,  and  recruitment/transfers. 

Inquiry,  sense  making,  common  grounding,  to  drive  down  levels  of 
uncertainty  and  reach  an  acceptable  level  of  confidence. 

Access,  availability,  permission,  provision,  preparation,  authority, 
certification,  and  use  related  to  equipment,  medications,  and  supplies. 

Assignment  of  ICT  team  members  to  best  match  patient  needs;  based  on 
individual  abilities  and  experience  and  team  needs. 

Identify,  confirm,  and  correct  information;  problem  detection,  which  may 
create  drag  in  completing  care  activities. 

Account  for  what  needs  to  be  done,  whether  it  has  been  completed,  and 
what  remains  to  be  done. 

The  ability  some  more  experienced  team  members  have  to  suspect 
something  that  is  needed  is  missing. 
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Collaboration  requires  the  expression  of  expectations,  prioritization,  and 
agreement  for  staff  member  recruitment  and  patient  transfers.  In  order  to 
reach  threshold  of  confidence  with  which  they  are  comfortable,  staff  mem¬ 
bers  clarify  through  inquiry,  sense  making,  and  seeking  common  care  by 
reducing  uncertainty.  Use  of  resources  such  as  equipment  depends  on  its 
availability  as  well  as  permission,  provision,  preparation,  authority,  and  any 
required  certification  to  use  them.  These  traits  fit  what  Cook  and  Woods 
(2002)  have  described  as  the  "technical  work"  in  the  context  of  health  care. 
Tasking  assigns  ICU  staff  members  to  best  match  individual  abilities/expe¬ 
rience  and  team  needs  to  meet  patient  needs.  Through  cross-checking,  the 
staff  detects  problems  and  identifies,  confirms,  and  corrects  information. 
Their  tracking  efforts  account  for  what  needs  to  be  done,  whether  it  has  been 
completed,  and  what  remains  to  be  done.  Staff  members  with  the  greatest 
expertise  are  able  to  see  "gaps,"  which  are,  in  effect,  "what  isn't  there"  but 
should  be. 


Challenges 

A  number  of  work  domain  issues  shown  in  Table  7.5  can  detract  from  the 
time  and  effort  that  could  be  devoted  to  patient  care.  Our  project  team  con¬ 
siders  each  issue  from  the  viewpoint  of  whether  the  cognitive  aid  could  help 
to  either  mitigate  or  eliminate  them.  Nurses  fill  gaps  in  the  limited  orienta¬ 
tion  that  residents  and  float  (off  unit)  nurses  receive,  which  takes  time  from 
patient  care.  Due  to  lags  in  information  timing  of  information  on  labs  and 
blood  cultures,  staff  members  need  to  rely  on  verbal  orders  (referred  to  as 
"on  the  sly")  that  are  not  fully  socialized  or  shared  and  can  result  in  care 
delays.  Bedside  nurses  reconcile  conflicts  between  patient  care  needs  and  tech¬ 
nology  protocols,  guidelines,  policy,  and  regulations.  Procedural  drag  results 
from  the  need  for  transcription  and  work-arounds  due  to  system  organiza¬ 
tional  gaps.  The  need  for  clinician  reliance  on  memory  provides  the  researcher 
with  a  marker  for  failure,  as  technology  fails  to  support  the  needed  work. 
The  long-term  story  of  the  patient/big  picture  is  lost,  because  trend  information 
and  understanding  are  lost  or  degraded  over  a  long  term  of  care.  Reliance 
07i  verbal  exchanges  makes  the  flow  of  information  porous,  brittle,  erratically 
shared,  and  less  reliable.  The  authority  gradient  between  junior  and  more 
senior  staff  members  encourages  passivity  with  regard  to  concerns  and 
impedes  sharing.  Common  grounding  accuracy  suffers  from  underspecifica¬ 
tion,  requiring  confirmation,  verification,  and  clarification.  It  is  not  always 
clear  who  has  the  "Con?"  (has  the  lead)  among  specialists  during  procedures 
when  care  quality  is  high,  but  no  individual  takes  accountability  to  assure 
results.  Timing  issues  can  result  in  poor  coordination  and  stale  information, 
such  as  when  a  procedure  was  performed.  Without  salience  to  bring  it  to  the 
clinician's  attention,  important  patient  information  such  as  "stat"  orders  is 
lost  in  homogenous  information  displays.  Software  usability/access/usefulness 
issues  result  in  difficulties  in  being  able  to  use  it,  having  the  knowledge  it 
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TABLE  7.5 

Emergent  Themes  of  Barriers  and  Challenges  to  Effective  Care 
Issue  Definition 


Limited  orientation 

Lags  in  information, 
medications,  labs, 
and  blood 
Bedside  nurse 
reconciles  conflicts 
Procedural  drag 

Reliance  on  memory 
as  a  failure  marker 

Story  of  the  patient/ 
big  picture  is  lost 
Reliance  on  verbal 
exchanges 
Authority  gradient 
Common  grounding 
accuracy 

Action/ who  has  the 
"Con?" 

Timing 


Salience 


Usability/ access/ 
usefulness 
Organizational 
issues  =  drag 


Residents  and  float  RNs  receive  limited  orientation  to  the  unit.  RNs 
provide  orientation,  which  takes  time  from  patient  care. 

Reliance  on  verbal  orders  "on  the  sly"  (informally)  that  are  not  fully 
socialized  or  shared;  creates  consistent  care  delays. 

Technology  protocol,  guidelines,  policy,  regulations,  and  patient  care 
needs  require  choices  to  be  made. 

The  need  to  create  work-arounds  and  bridging  tactics  to  fill  the  gap 
between  incompatible  systems  slows  down  work  efficiency. 

Technology  fails  to  support  necessary  work,  causing  clinicians  to  rely 
on  memory  for  continuity  (e.g.,  action  items  not  completed  by 
afternoon  rounds  not  carried  through  to  the  next  day). 

Incremental  views  of  patient  status  are  not  synthesized  into  a  whole 
picture;  particular  concern  for  patients  in  BICU  for  extended  periods. 

Information  flow  is  porous,  brittle,  not  shared,  or  reliable. 

Encourages  passivity  with  respect  to  expressing  concerns. 

Under  specification,  needs  for  confirmation,  verification,  clarification 
all  affect  ability  of  clinicians  to  develop  consensus. 

Numerous  well-qualified  clinical  specialties  collaborate  but  lack  of 
clarity  regarding  who  is  leading  a  particular  procedure  (e.g.,  ECMO). 

Lack  of  synchrony  can  result  in  stale  information  (e.g.,  when  the 
procedure  was  performed). 

Great  deal  of  information  that  is  presented  homogenously. 

Information  that  is  most  relevant  is  difficult  to  find  (e.g.,  "Stat" 
orders  are  not  evident). 

Systems  cannot  be  used  without  requisite  operator  knowledge,  certain 
access  requirements. 

Compliance  with  administrative  reminders  detracts  from  patient  care. 


requires  to  use  it,  and  being  able  to  enter  data  accurately.  Compliance  with 
organizational  issues  such  as  administrative  reminders  creates  drag  for  clini¬ 
cian  efficiency. 


Discussion 

The  ICU  Work  Setting 

ICU  patients  present  clinical  teams  with  unique  challenges  and  complex 
combinations  of  life-threatening  injuries  and  illnesses.  Care  for  this  patient 
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population  is  necessarily  multidisciplinary  and  includes  many  special¬ 
ties.  Care  providers  across  these  clinical  areas  must  collaborate  to  develop 
treatment  plans,  assess  progress,  and  refine  or  change  treatment  plans  and 
modes. 

Clinician  decisions  are  only  as  good  as  the  information  that  is  available 
when  they  are  made.  The  daily  work  on  the  unit  requires  representations 
that  serve  as  a  map  of  the  ever-changing  environment  of  work  that  must  be 
successfully  navigated.  Clinical  teams  that  care  for  ICU  patients  in  the  mili¬ 
tary  health  care  system  encounter  these  challenges  as  they  make  diagnostic 
and  therapeutic  decisions  and  share  them  with  colleagues.  Decision-making 
difficulty  increases  as  the  number  of  patients  and  the  severity  of  their  condi¬ 
tions  increase.  Complexity  grows  as  the  number  of  care  providers  seeks  to 
make  their  own  unique  contribution  to  a  patient's  care. 

Patient  care  activities  rely  on  the  acquisition,  portrayal,  and  analysis  of 
therapeutic  and  diagnostic  information  from  many  sources.  This  creates 
a  complex  work  setting  that  is  composed  of  multiple  independent  agents. 
All  interact  in  various  ways  according  to  inconsistent  rules  in  an  attempt  to 
adapt  to  changing  conditions.  Because  of  this,  the  organization's  outcomes 
are  unpredictable,  but  they  often  follow  predictable  patterns  (Plsek  and 
Greenhalgh  2001). 

Other  ethnographic  studies  also  revealed  insights  into  acute  care  settings. 
For  example,  Fackler  et  al.  (2009)  used  CTA  to  identify  cognitive  aspects  of 
critical  care  practice  in  two  academic  ICUs  and  identified  broad  categories  of 
cognitive  activity:  pattern  recognition;  uncertainty  management;  strategic  vs. 
tactical  thinking;  team  coordination  and  maintenance  of  common  ground; 
and  creation  and  transfer  of  meaning  through  stories.  Anders  et  al.  (2012) 
used  a  simulator-based  experiment  to  evaluate  ICU  nurses'  ability  to  detect 
patient  changes  using  an  integrated  graphical  information  display  (IGID) 
compared  with  a  conventional  electronic  chart-style  ICU  patient  informa¬ 
tion  display.  The  study  found  that  the  32  ICU  nurse  samples  reported  more 
important  physiological  information  with  the  novel  IGID  compared  with  the 
tabular  display  and  concluded  that  information  displays  should  accommo¬ 
date  the  diversity  of  those  who  are  intended  to  use  it. 

Novak  et  al.  (2012)  found  that  medication  administration  intersects  with 
other  organizational  routines,  and  IT-enabled  changes  to  one  routine  lead 
to  unintended  consequences  in  its  intersection  with  others.  Introducing  IT 
can  be  improved  by  nurses  who  provide  technology-use  mediation  before 
and  after  the  rollout  of  a  new  health  IT  system.  Their  efforts  can  help  others 
to  better  understand  the  relationship  between  IT  introduction  and  changes 
in  routines. 

In  addition  to  operational  complexity,  our  research  into  reporting  health 
care  adverse  events  using  CSE  methods  (Nemeth  et  al.  2006)  has  also  revealed 
technical,  social,  political,  and  legal  forces.  Each  influences  acute  care  set¬ 
tings  such  as  the  ICU,  which  are  typically  uncertain,  interrupt  driven,  satu¬ 
rated,  and  contingent. 
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Uncertain:  Clinicians  must  treat  widely  varying  patient  populations.  Time 
pressure  can  force  clinicians  to  make  decisions  based  on  information  that 
can  be  insufficient  or  ambiguous.  Field  studies  using  CSE  methods  can  dis¬ 
cover  initiatives  that  clinicians  have  developed  to  minimize  uncertainty. 

Interrupt  driven:  Interruptions  create  breaks  in  clinicians'  task-oriented 
work  (Chisholm  et  al.  2000),  and  when  they  occur  during  diagnosis  and  treat¬ 
ment,  they  can  degrade  or  defeat  attempts  to  treat  patients.  Work  domain 
study  using  CSE  methods  can  identify  gaps  in  care  continuity,  detect  how 
clinicians  allocate  limited  attention  reserves,  and  produce  tools  such  as  cog¬ 
nitive  artifacts  that  maximize  patient  care  opportunities. 

Saturated:  Facilities  and  staffs  typically  run  at  or  near  capacity.  With  little 
margin  of  time  or  resources  to  spare,  clinicians  have  to  develop  strategies 
to  cope  with  variations  in  care  demand.  Work  domain  studies  using  CSE 
can  reveal  discontinuities  that  exist  in  the  match  between  resources  and 
demand,  such  as  late  shifts,  and  unexpected  surges  in  care  demand. 

Contingent  The  process  of  care  depends  on  the  patient,  including  present¬ 
ing  symptoms,  documentation  of  history,  response  to  therapy,  expected  tra¬ 
jectory  of  treatment,  compliance,  and  more.  CSE  methods  can  be  used  to 
discover  how  care  providers  create,  monitor,  and  adjust  multiple  contingen¬ 
cies  in  order  to  achieve  as  satisfactory  and  expedient  an  outcome  as  possible 
for  patients. 

In  addition,  distraction,  complexity,  remote  influences,  and  consideration 
make  health  care  human  subjects  research  a  particular  challenge. 

Distraction:  Many  activities  are  performed  by  a  variety  of  clinicians  in  the 
vicinity  of  each  other.  This  makes  it  easy  to  be  distracted  by  phenomena  that 
are  not  necessarily  key  features  of  the  work  domain. 

Complexity:  Acute  care  settings  have  many  complex  activities  that  occur  at 
the  same  time.  This  is  particularly  true  in  an  ICU. 

Remote  influences:  Care  team  members  can  be  distributed  across  various 
locations  and  across  time.  Not  all  activity  that  matters  occurs  within  view  or 
in  the  immediate  recall  of  those  whom  the  researcher  interviews. 

Consideration:  Patients  in  the  BICU  are  typically  fragile  as  a  result  of  some 
trauma.  This  calls  for  the  researcher  to  have  an  adequate  sensitivity  to  care 
providers,  patients,  and  the  patient's  family  members. 

All  of  these  influences  form  the  context  in  which  clinicians  perform  their 
cognitive  work.  The  CSE  approach  makes  it  possible  to  describe  the  domain 
and  individual  and  team  activity  in  it  to  transform  findings  into  require¬ 
ments  that  serve  as  the  basis  for  a  prototype  cognitive  aid. 

Communication  among  Care  Team  Members 

Team  communication  creates,  and  is  created  by,  the  work  context.  CSE  can 
be  used  to  reveal  the  context  and  worker  behaviors  that  lead  to  understand¬ 
ing  communication  needs  and  how  to  support  them.  This  contrasts  with 
the  more  traditional  information  engineering  approach  that  assumes  that 
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understanding  comes  simply  from  the  faithful  uninterrupted  transmission 
of  data  (Feldman  and  March  1981;  Stohl  and  Redding  1987).  Care  provider 
expectations  differ  on  communication  content,  form,  relevance,  and  value  of 
its  completeness. 

Interventions  based  on  CSE  methods  can  benefit  team  communication.  For 
example,  Grome  et  al.  (2009)  found  that  co-creative  development  workshop 
helped  surgical  team  representatives  to  create  and  adapt  preoperative  brief¬ 
ing  content  and  structure,  as  well  as  measures  to  assess  the  briefing's  effect 
on  teamwork,  communication,  and  patient  safety. 

Nemeth  and  Cook  (2013)  used  CSE  to  identify  barriers  that  can  erode 
the  quality  and  reliability  of  health  care  communication  that  this  project 
addresses. 

Difficulties  in  communication.  Health  care  and  the  information  that  is  needed 
to  provide  it  are  typically  complex  and  demand  accuracy  in  order  to  avoid 
misinterpretation. 

Confusion  of  responsibility.  Interwoven  relationships  among  care  provid¬ 
ers,  units,  departments,  and  institutions  can  result  in  confusion  over  who  is 
responsible  for  a  patient's  care. 

Lack  of,  or  variable  availability  of,  good  information  resources.  Even  with  sophis¬ 
ticated  information  technology  available,  system  failure  or  incompatibility 
can  result  in  images  and  reports  being  mislabeled,  misunderstood,  swapped, 
late,  misidentified,  or  unavailable. 

Work  environment  pressures.  Care  provider  efforts  to  cope  with  workload 
demands  and  time  pressure  can  result  in  a  kind  of  "shorthand"  that  edits 
information  in  order  to  be  efficient. 

Lack  of  standards  or  training.  Clinical  specialties  and  institutions  can  vary  in 
the  way  they  go  about  practices  such  as  handoffs,  resulting  in  the  potential 
for  misperception. 

Aptitude.  Patients  and  family  members  may  find  it  hard  to  understand  the 
information  that  is  conveyed  through  written,  verbal,  and  graphic  health 
care  communication. 

Attention.  Understanding  and  context  are  essential  to  effective  communi¬ 
cation.  Simple  transmission  (e.g.,  a  "data  dump")  does  not  guarantee  that 
others  understand  what  is  provided  or  can  correctly  put  it  into  context. 

Attitude.  Clinician  empathy  may  yield  a  number  of  benefits,  including 
patients  reporting  more  about  their  symptoms  and  concerns,  increased 
physician  diagnostic  accuracy,  patients  receiving  more  illness-specific  infor¬ 
mation,  increased  patient  participation  and  education,  increased  patient 
compliance  and  satisfaction,  greater  patient  enablement,  and  reduced  patient 
emotional  distress. 

Reader  et  al.  (2008)  found  that  team  structure  and  individual  roles  and 
stature  have  significant  effect  on  ICU  communication,  and  a  difference  in 
status  appears  to  influence  how  communication  is  perceived-  The  "authority 
gradient"  barrier  mentioned  in  Table  7.5  may  be  related  to  this  issue. 
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Through  the  use  of  CSE,  the  cognitive  aid  that  this  project  produces  will 
need  to  help  the  ICU  staff  to  overcome  these  potential  barriers. 


The  Role  of  CSE 

The  use  of  CSE  methods  makes  it  possible  for  the  researcher  to  "get  in"  at  the 
right  level  of  detail.  Too  general  a  study  will  miss  the  nuances  and  refine¬ 
ments  that  clinicians  create  in  order  to  make  their  work  possible.  Too  detailed 
a  study  may  collect  great  amounts  of  data  but  will  also  miss  the  broader  pat¬ 
terns  that  make  insight  possible.  Studies  of  such  a  complex  domain  require 
repeated  visits  in  order  to  reveal  the  deeper  aspects  of  what  occurs.  These 
are  what  have  been  referred  to  as  the  "messy  details"  of  technical  work 
(Nemeth  et  al.  2004).  The  researcher  needs  to  learn  about  real-world  settings 
that  involve  the  organized  activities  of  daily  life  (Garfinkel  1967).  Real-world 
settings  are  stubborn,  though,  and  do  not  easily  reveal  themselves  (Blumer 
1969). 

Research  can  be  basic  (a  search  for  general  principles),  applied  (adapting 
general  findings  to  classes  of  problems),  or  clinical  (related  to  specific  cases). 

Most  design  research  is  clinical  because  time  and  budget  allow  for  little  else 
(Friedman  2000).  CSE  methods  can  be  used  to  negotiate  the  gap  between 
applied  and  clinical  research. 

CSE  in  Health  Care 

Recent  work  on  collaboration  has  produced  distributed  cognition  and  joint 
cognitive  system  models  that  can  be  used  to  better  understand  health  care 
as  a  collective  enterprise.  The  use  of  CSE  to  identify  and  describe  all  ICU  ele¬ 
ments,  including  clinicians,  information,  and  artifacts,  can  identify  system 
gaps.  Addressing  gaps  can  lead  to  authentic  improvement  in  performance 
and  outcomes.  For  this  reason,  CSE  is  particularly  well  suited  to  the  discov¬ 
ery  of  phenomena  in  complex  real-world  settings. 

Distributed  cognition  (Hutchins  1995)  is  the  interaction  of  individuals, 
artifacts,  and  the  environment.  Practitioners  must  rely  on  this  to  prevent  the  "Practitioners 

r  .  r  *1  •  •  r  ,  n»r*/'x/A\  ti  •  •  i  i  must  rely  on  this 

formation  ot  gaps  m  the  continuity  ot  care  (Cook  et  al.  zUuu).  this  includes  to  prevent  the 

ri  l  ,  .  '  .  .  .  .  ,  formation  of  gaps 

transfers  between  departments,  work-cycle  shift  changes,  and  information  in  the  continuity 

r  J  r  i  i  r  of  care  (Cook  et 

exchanges  among  professionals  from  different  fields  ot  practice.  Clinicians  a/2ooo>"  Edits 
in  an  ICU  comprise  a  joint  cognitive  system  that  can  modify  its  behavior 
and  decision  making  on  the  basis  of  experience  in  order  to  maintain  order 
(Hollnagel  and  Woods  1983).  The  daily  work  of  the  clinician  requires  rep¬ 
resentations  that  serve  as  a  map  of  the  ever-changing  environment  of  work 
that  must  be  successfully  navigated  (Rasmussen  et  al.  1994).  Individual  ele¬ 
ments  of  information  vary  enormously  in  the  length  of  time  that  they  are 
reliable,  and  their  value  depends  on  their  context.  What  is  represented  and 
how  it  is  represented  should  depend  on  the  cognitive  work  it  is  intended  to 
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support.  Furthermore,  the  partial  and  overlapping  interaction  among  clini¬ 
cal  specialties  in  the  ICU  lends  itself  to  additional  gaps  in  care  continuity  and 
the  misadventures  that  can  result. 


Validity 

Nemeth  et  al.  (2011)  recommended  four  ways  to  verify  whether  results  from 
qualitative  studies  such  as  this  ICU  research  project  are  valid.  Findings  must 
be  credible,  consistent,  comprehensive,  and  central. 

Credible.  Do  findings  "ring  true"  to  SMEs  and  others  who  work  in  the 
domain? 

Consistent.  Do  findings  replicate  across  interviews  and  across  incidents? 

Comprehensive.  How  general  are  the  findings?  To  what  range  of  tasks  and 
settings  do  they  apply?  Can  boundaries  be  identified,  and  can  those  limita¬ 
tions  be  stated? 

Central.  Do  findings  speak  to  cognitive  issues  that  matter  for  performance 
based  on  SME  judgments,  research  literature,  and  other  sources? 

Studies  that  meet  these  criteria  are  more  likely  to  pass  validity  tests  when 
solutions  are  evaluated. 


Aspects  of  Resilience 

Knowledge  gained  through  the  use  of  CSE  about  the  nature  of  work  as  it  is 
actually  done  can  help  to  contribute  to  the  system's  ability  to  adapt  when 
confronted  with  unforeseen  challenges — to  be  more  resilient  (Hollnagel  et 
al.  2006).  Recent  writing  in  resilience  engineering  has  identified  a  number 
of  system  characteristics  that  contribute  to  system  resilience.  This  knowl¬ 
edge  can  improve  their  ability  to  operate  despite  significant  challenges  such 
as  changes  in  the  type,  rate,  and  volume  of  care.  Three  characteristics  that 
CSE  can  assist  include  being  self-aware,  the  ability  to  identify  and  apply 
resources,  and  the  ability  to  adapt  to  surprise. 


Self-Aware 

The  entry  “Reid 

The  "cottage  industry  structure  of  the  national  healthcare  delivery  system"  etai.2005"is 

~  J  tu-A  i  not  included  in 

results  in  'disconnected  silos  of  function  and  specialization.  (Reid  et  al.  the  reference 

r  x  list.  Please 

2005,  pp.  12-13)  Acute  and  ambulatory  care  patients  require  coordinated  care  provide  missing 

11  J  1  1  information  in  the 

that  is  provided  by  multiple  distributed  care  providers.  Their  care  also  calls  references 
for  the  coordination  and  integration  of  many  functions  and  specialized  areas 
of  knowledge  over  time.  Yet  connectivity,  integrated  care,  and  coordination 
are  inadequate  nationwide  at  all  stages  of  illness  treatment.  An  estimated 
60  million  patients  in  the  United  States  suffer  from  two  or  more  chronic 
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conditions  and  are  particularly  affected  by  the  disconnection  among  clinical 
care  specialties.  The  ability  to  reveal  the  nature  of  work  domains  by  using 
CSE  can  start  to  mitigate  this  significant  and  widespread  issue. 


Able  to  Identify  and  Apply  Resources 

Skills,  supplies,  equipment,  and  facilities  are  routinely  assembled  to  perform 
each  procedure.  CSE  can  be  used  to  document  work  processes  and  what 
influences  them.  That  can  lead  to  insight  into  how  these  configurations  are 
developed  and  managed,  what  goes  well,  and  where  misadventures  can 
occur. 


Able  to  Adapt  to  Surprise 

We  have  shown  in  prior  publications  (Nemeth  et  al.  2007;  Cook  and  Nemeth 
2010)  how  health  care  organizations  respond  to  events,  particularly  misad¬ 
ventures.  More  often  than  not,  the  response  attempts  to  isolate  the  cause  and 
declare  that  it  will  not  happen  again.  These  efforts  stop  the  exposure  to  risk. 
However,  they  also  stop  the  learning  that  can  inform  us  how  systems  have 
difficulty  adapting.  The  use  of  CSE  makes  understanding  what  goes  right, 
and  what  occasionally  does  not,  a  routine  learning  process  that  can  improve 
the  ability  to  adapt. 


Summary 

We  need  to  learn  what  people  actually  do  in  health  care  teams  and  how  to 
design  work  processes  and  systems  based  on  that  knowledge.  This  calls  for 
an  approach  that  reveals  the  true  nature  of  work  as  it  is  actually  done,  not  as 
it  is  intended  to  be  done.  CSE  serves  that  purpose  well. 

Early  data  collection  and  analysis  activity  in  our  BICU  research  have  iden¬ 
tified  the  network  of  those  who  care  for  patients,  the  information  sources 
they  use,  and  the  flow  of  patient  care  activity.  Continued  visits  are  expected 
to  deepen  the  understanding  of  interrelationships  among  clinicians,  how 
they  address  and  resolve  conflicts  such  as  different  agendas,  the  information 
sources  and  their  use,  and  cognitive  activities  for  each  of  the  clinical  special¬ 
ties  and  roles.  Results  from  this  first  year  of  study  will  be  used  to  develop 
requirements  for  decisions  that  clinicians  make.  Requirements  and  use  cases 
will  provide  the  basis  for  a  prototype  to  be  developed  and  evaluated  in  the 
project's  second  and  third  years. 

The  well-designed  valid  cognitive  artifact  that  results  from  our  use  of 
CSE  is  intended  to  support  individual  and  team  cognitive  work,  which  is 
expected  to  improve  the  reliability  and  efficiency  of  clinical  care  for  patients. 
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Abstract — Sensitivity  to  patient  needs  makes  clinicians  the 
primary  source  of  adaptive  capacity,  or  resilience ,  in  the 
intensive  care  unit  (ICU).  Work  setting  complexities  and 
contingencies  make  cognitive  work  in  this  setting  particularly 
challenging.  A  IT-based  system  to  support  individual  and  team 
decisions  and  communication  would  increase  clinicians'  capacity 
to  adapt.  We  report  on  a  3-year  project  now  underway  to  develop 
such  a  system.  During  the  first  year,  our  research  team  used 
Cognitive  Systems  Engineering  (CSE)  methods  to  reveal 
characteristics  of  the  work  setting,  goals,  barriers,  and  individual 
and  team  initiatives  to  overcome  barriers.  Our  data  analyses 
identified  requirements  for  the  IT  system  that  were  embodied  in 
use  cases,  as  well  as  in  first  draft  prototypes  of  the  system 
architecture  and  user  interface.  Our  team  is  currently  evaluating 
the  interface  prototype  for  face  validity  and  refining  details  prior 
to  starting  programming.  Interactive  prototypes  will  be 
evaluated  against  criteria  identilied  in  field  research  to  ensure 
validity.  The  resulting  system  is  expected  to  improve  staff 
decision  making  ability  and  communications  with  an  expected 
improvement  in  unit  adaptability.  Shared  decisions  based  on 
belter  information  about  procedures  and  resources  are  expected 
to  improve  staff  efficiency  and  decrease  missteps,  lapses,  delays 
in  care,  and  the  occurrence  of  morbidities  including  wrong 
medication/dose,  infections,  and  unanticipated  emergencies  such 
as  cardiac  arrest. 

Keywords — cognition,  decision  support,  communication, 

healthcare 

Introduction 

The  U.S.  Department  of  Defense  maintains  one  of  the 
largest  healthcare  networks  in  the  world.  It  provides  in-patient 
and  out-patient  care  for  the  active  military,  their  families, 
reserve  forces,  veterans,  and  local  civilians  through  various 
military  healthcare  centers.  Caring  for  patients  who  are 
admitted  to  Intensive  Care  Units  (ICUs)  presents  healthcare 
teams  with  unique  challenges  that  stem  from  patients’  fragile 
condition  and  the  complex  combination  of  hfe-threatening 
injuries  and  illnesses  they  face. 
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Care  for  ICU  patients  necessarily  depends  on  collaboration 
by  staff  members  from  a  number  of  healthcare  disciplines  and 
relies  on  clinician  decision  making  and  related  activities, 
which  is  termed  cognitive  work.  Care  providers  among 
multiple  professions  must  work  together  to  make  effective 
decisions,  develop  treatment  plans,  assess  patient  progress, 
and  refine  care  management  over  lime.  However,  their 
decisions  are  only  as  good  as  the  information  that  is  available 
and  evident  when  the  decisions  are  made.  For  this  reason,  the 
Institute  of  Medicine  [1]  has  recommended  improving  access 
to  accurate,  timely  information,  and  making  relevant 
information  available  at  the  point  of  patient  care. 

Computer  systems  and  knowledge  resources  are  available 
to  support  cognitive  work,  but  gaps  among  these  resources  and 
among  care  providers  cause  difficulties  in  healthcare  delivery 
As  a  result,  critical  information  that  is  needed  to  make 
decisions  is  difficult  to  obtain,  is  often  unavailable  when  it  is 
needed  most,  and  is  difficult  to  share. 

THE  COOPERATIVE  COMMUNICATION  SYSTEM 

Healthcare  providers  and  related  sources  of  information 
including  information  systems,  equipment  displays  and  more 
comprise  a  joint  cognitive  system  [-J  that  is  used  to  manage 
care  activities.  Our  research  team  is  developing  a  Cooperative 
Communication  System  (CCS)  that  will  serve  as  part  of  the 
joint  cognitive  system  in  a  16-bed  military  tertiary  care  Burn 
ICU  (BICU).  The  CCS  is  expected  to  enable  the  healthcare 
team  to  remain  connected  to  an  individual  patient,  patient 
information,  and  to  each  other  across  time  and  location  as  the 
team  delivers  care.  It  will  keep  providers  informed  of  a 
patient’s  status,  and  of  other  healthcare  providers’  patient  care 
activities,  enable  the  staff  to  understand  goals,  objectives  and 
tasks  related  to  each  patient,  and  to  reconcile  differing  points 
of  view.  The  decision  support  that  the  CCS  provides  will 
make  it  possible  for  clinicians  to  make  more  accurate  and 
timely  diagnoses,  order  more  timely  and  appropriate  tests,  and 
make  better  plans  so  that  patients  receive  better  care.  Use  of 
the  CCS  is  expected  to  improve  patient  outcomes  by 
improving  the  availability  of  information,  and  the 
synchronization  of  care  among  BICU  team  members. 
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The  CCS  project  is  organized  in  three  phases.  Phase  1 
collected  and  analyzed  data  to  understand  the  cognitive  work 
and  barriers  to  effective  patient  care.  Results  were  used  to 
develop  CCS  system  design  requirements.  Phases  2  and  3  will 
develop  a  prototype  CCS  system  and  a  test  bed  based  on  the 
BICU  clinical  environment  that  will  be  used  to  evaluate  the 
CCS  system  with  clinicians. 

METHODS 

Descriptions  of  clinical  cognitive  activities  rely  on 
understanding  how  individuals  and  groups  perform  them  in  an 
actual  (“field")  work  setting  Field  research  requires 
immersion  to  enable  the  researcher  to  observe  actual  work 
practice  and  gain  insight  from  deep,  repeated  inquiries  [3]. 

A.  Human  Subject  Research  Approval 
Before  any  data  were  collected,  the  research  team  obtained 
approval  for  human  subject  research  from  the  funder  and 
research  site  Institutional  Review  Board.  A  total  of  151  staff 
members  consented  to  participate. 

13.  Cognitive  Systems  Engineering 

Understanding  any  work  domain  and  the  forces  that  shape  it 
requires  methods  that  are  suited  to  its  study.  The  project  team 
is  using  a  Cognitive  Systems  Engineering  (CSE)  [4,  5]  mixed 
methods  research  approach,  which  is  particularly  well-suited 
to  study  cognitive  activity  in  field  settings  such  as  the  Bum 
ICU.  Cognitive  Systems  Engineering  is  the  process  of  learning 
about  behavior  and  cognition  as  humans  confront  complexity 
in  their  work  settings,  and  providing  tools  to  support  their 
behavior.  The  CSE  approach  is  used  to  translate  knowledge 
about  human  cognitive  performance  such  as  what  is  needed  to 
attract  attention  to  unexpected  data  into  prmciples  and 
techniques  to  develop  solutions  including  human-computer 
interface  design.  [6] 

Fig  1  illustrates  five  phases  in  the  approach  and  how  the 
activities  in  each  map  to  phases  of  this  project. 


Fig  1 .  Five  phases  of  Cognitive  Systems  Engineering.  Adapted  from  [7J 

As  “systems  engineering,”  the  CSE  approach  includes  all 
agents  that  can  act  in  the  work  setting,  such  as  a  Bum  ICU  that 
is  being  studied.  As  Fig.  1  shows,  CSE  phases  span  data 


collcctioa  data  analysis,  and  solution  development. 
Integration  of  these  five  phases  ensures  that  the  solution  the 
process  produces  is  grounded  in  worker  and  work  setting  data. 
The  ability  to  identify  each  element  among  workers,  work 
setting,  and  tools  can  also  help  designers  to  anticipate  shifts 
and  unintended  coasequences  that  occur  when  new 
information  technology  (IT)  such  as  the  CCS  is  introduced  [8], 

During  the  first  year,  the  project  team  collected  data  over 
10  months  in  four  week-long  visits  to  the  Bum  ICU.  During 
each  visit,  the  team  conducted  formal  interviews,  observed 
and  shadowed  clinicians,  and  documented  artifacts  such  as 
paper  forms,  infomiation  systems,  and  displays  that  the  staff 
uses  to  help  them  accomplish  their  work.  The  team’s  research 
nurse  helped  to  collect  data  when  the  team  was  not  at  the 
research  site.  Following  each  site  visit,  team  members  met  to 
analyze  the  data  over  multi-day  analysis  sessions.  Data 
analysis  involved  several  iterative  steps.  The  team  reviewed 
and  discussed  data  multiple  times  to  understand  it  thoroughly, 
identify  gaps,  reduce  data,  and  synthesize  it  into  findings. 

The  team  started  with  structured  and  systematic  passes 
through  the  data  to  delect  patterns,  or  themes,  which  described 
both  the  ICU  work  setting  and  clinician  cognitive  work.  The 
team  used  the  themes  they  had  developed  during  the  team 
working  sessions  to  code  interview  transcripts  and  observation 
notes  that  identified  relevant  portions  for  each  theme.  After 
data  coding,  each  research  team  member  was  assigned  a 
subset  of  the  coded  data  excerpts  to  review  and  interpret.  The 
team  held  another  two-day  working  session  to  synthesize  and 
integrate  findings.  Following  the  synthesis,  the  team  created 
initial  requirements  for  CCS  according  to  barriers  clinicians 
face  and  what  the  CCS  system  could  do  to  help  clinicians  to 
overcome  them.  The  team  then  presented  the 
challenges/barriers  and  initial  requirements  to  two  physicians 
and  three  nurses  on  the  unit  to  get  their  initial  appraisal  of  the 
findings’  face  validity. 

The  analysis  provided  the  means  to  identify  Burn  ICU 
clinical  team  cognitive  work  requirements.  The  team  also 
closely  reviewed  the  forms  and  documents  that  the  Burn  ICU 
clinical  teams  use  to  understand  the  kinds  of  information  they 
seek,  use,  and  share  with  one  another.  They  developed  models 
of  the  BICU  work  domain  and  clinician  decision-making  and 
patient  care  through  this  process  that  described  the  unit’s 
information  content  and  flow  that  the  prototype  CCS  system 
will  help  to  manage. 

FINDINGS 

The  project  team  identified  20  key  challenges  and  barriers  to 
cognitive  work  on  the  BICU,  then  translated  them  into  concise 
problem  statements  and  information  system  requirements. 
They  developed  representations  to  describe  the  BICU 
environment  and  key  resources  that  clinicians  use  there, 
formulated  a  set  of  use  cases  to  describe  to  developers  how  the 
system  is  intended  to  work,  and  developed  an  initial 
descriptive  model  of  Bum  ICU  cognitive  work  (Fig.  2). 
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Members  Perform 


Activities  Unit  Tasks  Unit 

Members  Perform  Members  Perform 


Clarification 


Reduce  uncertainty 
Manage  ambiguity 
Manage  conflicting  agendas 
Manage  communication 


Synchronization 


Coordination — 


Develop  shared  awareness  of  the  Patient 
Get  and  keep  common  ground 
Manage  the  care  plan  and  treatment  goals 
Coordinate  resources 


Negotiation 


Identify  alignment,  gaps/differences 
(e.g„  agendas) 

Manage  conflicting  agendas 
Manage  communication 


Anticipation - 1 


Forward  thinking 


Fig  2.  Model  of  cognitive  work 
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A.  Model  of  Cognitive  Work 

Complexity  can  hide  underlying  systematic  patterns  in 
cognitive  work.  Fig.  2  illustrates  these  patterns  in  the  BICU. 
Synchronization  of  patient  care  among  clinicians  and  over 
time  is  the  top  level  of  the  model.  The  next  level  down 
includes  activities  that  all  unit  members  perform: 
clarification,  coordination,  negotiation,  and  anticipation, 
followed  by  supporting  tasks.  Bach  task  can  be  observed  in  the 
way  that  clinicians  interact  with  each  other  and  use 
information  sources  to  minimize  uncertainty.  Requirements 
that  the  team  developed  from  these  tasks  indicate 
opportunities,  or  leverage  points,  to  improve  synchronization. 

B.  Patient  Care  Providers 

Part  of  the  challenge  in  this  project  is  to  know  how  to  bound  it. 
To  do  that,  the  team  asked  “Who  do  you  communicate  with  to 
do  your  work?”  of  8  nurses,  5  respiratory  therapists,  2  physical 
therapists  /occupational  therapists,  1  nutritionist,  and  1 
physician.  Fig.  3  shows  the  resulting  network  that  can  be  used 


Fig  3.  Care  provider  relationships  closest  lo  patient 
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to  develop  the  CCS  interface  structure.  Thicker  lines  show  that 
communication  was  mentioned  by  both  parties,  and  enable  the 
team  to  organize  interface  screens  according  to  clinical  roles. 
This  initial  network  will  expand  as  project  work  continues, 
providing  the  basis  for  interface  views  that  are  organized 
according  to  BICU  work  roles. 


C.  Information  Sources 

The  team  identified  a  range  of  information  sources  (shown  in 
Fig.  4)  that  need  to  be  used  together  to  manage  care  and 
manage  the  ICU.  Ten  are  computer-based,  3  are  paper  artifacts, 
and  3  are  computer-based  displays  that  produce  a  paper 
printout.  Communications  including  cell  and  land  line  phones 
and  email  are  further  information  sources.  The  set  describes  an 
inventory  of  information  that  matters  to  the  clinicians,  and  each 
needs  to  be  included  in  the  CCS  solution. 
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Fig  4.  Information  sources  clinicians  use  on  the  BICU 
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D.  Barriers 

Bach  of  the  barriers  the  team  discovered  presents  an 
opportunity  to  ask  how  the  CCS  system  can  help  to  improve 
unit  synchronization.  Here  are  four  of  the  20  barriers  that  the 
team  discovered: 

No  effective  means  exists  to  synchronize  aspects  of 
patient  care. 

There  is  a  lack  of  awareness  of  activities  and  events  that 
are  tightly  coupled 

There  is  no  efficient  way  to  communicate  changes  in 
patient  status  across  clinical  specialties. 

Updated  information  such  as  results  of  laboratory  cultures 
is  available  but  is  not  accessible  or  visible. 

E.  Requirements 

Using  the  challenges  and  barriers,  the  team  created  a  set  of 
problem  statements  and  then  developed  concise  statements  of 
system  requirements  for  each.  The  first  barrier  provides  an 
example: 
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A 'o  effective  means  to  synchronize  and  adapt  different  aspects 
of patient  care  over  the  course  of  a  shift,  across  caregiver 
team. 

The  requirement  states  how  the  CCS  solution  can  help  to 
overcome  the  barrier: 

System  shall  provide  access  to  a  plan  of  patient  care,  visible  to 
all  care  givers  responsible  for  that  patient  that  includes: 

Current  patient  status  and  top-level  assessment; 

Goals  and  priorities  for  those  goals; 

Changes/updates,  such  as  indication  that  plan  is  being 
updated  when  one  caregiver  is  working  on  it; 

Schedule  of  activities  and  any  changes,  timeline; 

Orders  and  their  status; 

Identity  and  contact  information  for  patient's  care  team 

The  requirement  starts  to  describe  the  CCS  interface's  content 
and  operation.  The  complete  set  of  requirements,  which  is 
directly  related  through  the  analyses  to  the  original  data,  can 
be  used  to  create  a  series  of  use  case  scenarios. 

F.  Use  Case 

The  first  paragraph  of  a  use  case  for  the  above  requirement 
describes  how  each  of  these  features  (shown  in  bold  type) 
would  serve  clinician  needs. 

At  0630,  a  bedside  nurse  has  started  his  preparation  for 
the  day  shift  by  reviewing  information  on  the  patient  he  is 
responsible  for.  Opening  CCS,  he  can  see  a  roster  of 
patients  on  the  unit,  chooses  his  patient’s  “at-a-glance” 
view  that  shows  recent  vital  signs,  current  orders, 
medications,  care  plan,  and  notes  from  the  night  shift. 
He  checks  the  patient’s  standing  care  plan  and  treatment 
goals  (from  the  electronic  healthcare  record),  and 
reviews  orders  (from  the  laboratory  test  database)  that 
are  pending  as  well  as  the  day 's  care  activities  that  the 
Wound  Care  team,  Respiratoty  Therapists,  and  Physical 
Therapists  have  recommended  and  what  times  they  can 
perform  them. 

The  information  designer  and  programmers  will  use  these 
requirements  and  use  cases  to  develop,  evaluate,  and  refine 
prototypes  in  Phases  2  and  3. 

DISCUSSION 

Methods  from  CSE  can  be  used  to  learn  the  nature  of  work  as 
it  is  actually  done,  and  when  it  is  done,  by  those  who  do  it. 
This  makes  it  possible  to  create  effective  solutions  that 
workers  recognize  and  readily  accept.  Using  knowledge  about 
a  work  setting  such  as  the  Bum  ICU  can  improve  workers' 
ability  to  operate  in  spite  of  significant  challenges  such  as 
unexpected  changes  in  the  type,  rate,  and  volume  of  care 
demand  [9],  Insights  from  such  studies  can  also  help  to 
contribute  to  the  system’s  ability  to  adapt — to  be  more 
resilient  [10] — when  workers  are  confronted  with  unforeseen 


challenges. 

Three  characteristics  that  CSE  can  assist  include:  being 
self-aware,  the  ability  to  identify  and  apply  resources,  and  the 
ability  to  adapt  to  surprise. 

A.  Self-Aware 

The  “cottage  industry  structure  of  the  national  healthcare 
delivery  system"  results  in  “disconnected  silos  of  function  and 
specialization"  [11],  This  disconnection  among  specialties  is 
aggravated  by  disconnected  information  sources.  In  this 
BICU,  for  example,  the  electronic  healthcare  record  is  not 
connected  to  the  outpatient  record  or  the  database  that  tracks 
laboratory  test  results.  Coping  with  these  gaps  forces 
clinicians  to  invent  their  own  “workarounds."  One 
workaround  is  to  read  a  display  on  one  system,  write  needed 
information  onto  a  scrap  of  paper,  walk  to  another  system 
display,  then  key  in  the  information.  The  process  not  only 
opens  the  door  to  transcription  error,  but  also  takes  away  from 
time  that  could  and  should  be  spent  caring  for  the  patient. 

The  CCS  can  contribute  to  ICU  self-awareness  by  bridging 
the  many  databases  that  are  currently  unconnected.  The 
synthesis  of  information  sources  would  also  open  the  way  for 
data  mining  to  seek  and  extract  meaningful  patterns  of 
information  that  are  related  to  the  patient,  the  unit,  or  the 
clinician(s). 

B.  Able  to  Identify  and  Apply  Resources 

Clinical  skills,  supplies,  equipment,  and  portions  of  facilities 
are  routinely  assembled  to  perform  each  Bum  ICU  procedure. 
Patient  condition  and  readiness  for  a  procedure  can  change, 
and  clinicians,  equipment,  or  rooms  can  become  available  or 
unavailable.  Scheduling  is  currently  done  using  hard  copy 
forms  and  in-person  negotiation,  which  makes  it  difficult  to 
develop  and  maintain  an  optimal  plan. 

The  CCS  can  improve  the  ability  to  identify  and  apply' 
resources  through  scheduling  that  supports  both  planning  and 
re-planmng  (making  changes  to  plans  as  the  day  progresses). 

C.  Able  to  Adapt  to  Surprise 

We  have  shown  in  prior  publications  [12,  13]  hew  healthcare 
organizations  respond  to  events,  particularly  misadventures. 
With  insufficient  information  on  what  actually  occurs,  the 
response  attempts  to  isolate  the  cause  and  declare  that  it  will 
not  happen  again.  A  system  that  can  adapt  to  surprises  and 
challenges  can  also  be  used  to  leam  about  its  response.  The 
use  of  CSE  makes  understanding  what  goes  right,  and  what 
occasionally  does  not,  a  routine  learning  process  that  can 
improve  the  ability  to  adapt. 

Data  mining  being  developed  for  the  CCS  will  make  it 
possible  to  detect  and  illustrate  trends.  Understanding  how  a 
patient  or  group  of  patients  fares  over  time  can  improve 
clinicians’  ability  to  adapt  to  surprises  such  as  unexpected 
changes  in  patient  condition. 
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SUMMARY 

Health  IT  has  significant,  pervasive  effects  on  health  care 
delivery,  patient  safety,  and  care  quality.  Methods  within  the 
of  CSE  approach  can  be  used  to  identify  patient  care  and  work 
setting  complexities  that  affect  clinicians  and  the  decisions 
they  make.  That  understanding  can  be  used  to  develop 
requirements  for  computer-based  cognitive  aids  to  improve 
individual  and  team  decision-making  and  communication. 

The  system  that  the  CCS  project  produces  is  expected  to 
improve  clinical  decision  making  and  communication  as  well 
as  unit  adaptability.  Shared  decisions  based  on  better 
information  about  procedures  and  resources  are  expected  to 
improve  staff  efficiency.  The  CCS  system  is  eventually 
expected  to  help  to  decrease  missteps,  lapses,  delays  in  care, 
and  the  occurrence  of  morbidities  including  wrong 
medication/dose,  infections,  and  unanticipated  emergencies 
such  as  cardiac  arrest.  As  the  study  continues,  the  research 
team  will  design  and  develop  a  prototype  that  can  also  mine 
data  for  relevant  information,  then  test  and  validate  the 
prototype  usmg  criteria  from  the  first  year  of  research 
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Background  -  The  problem 

■  In  2009  we  were  frustrated  with  our  electronic  health  record  and 
began  seeking  answers  as  to  why 

■  At  that  time,  we  discovered  there  was  little  apparent  science  into 
how  EHRs  were  designed 

■  More  importantly,  our  clinicians  still  longed  for  the  large,  paper- 
based  flow-sheets  of  ICUs  since  past....  a  foolish  dream? 

■  So,  we  asked,  what  should  we  display  on  an  electronic 
representation  of  the  data  we  once  found  so  easily  in  an  paper 
flow  sheet? 
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Complexity  in  Critical  Care 


Complex 

Multiple  independent  agents 
Emergent,  non-linear  interaction 
Adaptable 


Dense 

Large  amounts  of  data 
Confined  space  and  time 
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Data  volume  before  and  During  ICU 

Total  data  points  per  patient-hour 
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Microbiology,  labs,  medications,  chest  X-ray,  Nurses  flowsheet,  Clinical  notes  (history  and 
impression/plan)  -  Vitals  excluded 


*  Slide  courtesy  of  Dr.  Vitaly  Hersavich,  Mayo  Clinic 
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Where's  the  patient? 
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Who's  in  Charge?  When? 
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_ In’s  &  Out’s 

At  least  8  different 
computer  screens! 

**  Not  shown:  Radiology, 

TEG,  CHCS  labs,  outpatient 
record,  “foreign”  records, 
images,  etc. 
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Labs 
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The  system  of  care 
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Objectives 

Overall  Activity  Objectives 

To  improve  care  by  supporting  clinical  decision-making  by  identifying  design 
requirements  for  computerized  cognitive  aides  and  communication  tools. 

Identified  Problem 

Traditional  scientific  methods  in  medicine  do  not  "unpack"  the  healthcare  system 
well. 

This  results  in: 

False  assumptions 

Limited  understanding  of  the  work  domain 


Cognitive  systems  engineering  has  demonstrated  effectiveness  in  understanding 
other  clinical  domains  such  as  the  operating  room,  emergency  department,  and 
the  pediatric  ICU* 

*  Nemeth  et  al.  IEEE  Transactions  on  Systems,  Man,  and  Cybernetics,  Part  A.  (2003)  vol.  34  pp.  726-735 

*  Nemeth,  et  al.  (2003).  A  Study  of  Pediatric  Intensive  Care  Unit  Technical  Work.  The  University  of  Chicago. 
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Cognitive  Systems  Engineering  Process  Molds  Result 


Understanding 
clinician  cognitive 
work  in  response 
to  challenges 
molds 

requirements  for 
solutions  such  as 
computer-based 
cognitive  aids 


II  M  II 


225! 


Tracing  Cognitive  Challenges  To 
Information  Requirements 


Field  data  collection 
leads  to 

Descriptive 
cognitive  models,  to 

Decision  and 
information 
requirements,  to 


Prototypes  to  be 
evaluated  and 
optimized 


13 


Page  92  of  132 


W81XWH-12-C-0126 


Research 

Design 


Conclusions 

Represent  the  na¬ 
ture  of  patient  care 
as  a  socio-cogni- 
tive  system 

Synthesis  & 
Evaluation 

Develop  &  test 
representations  of 
domain  semantics 


Findings 

Statements  regarding 
the  nature  ot  clinical  care 
and  representations 


Evaluation 

Determine 
suitability  of 
representations 
to  aid  clinician 
cognitive  work 


Analysis 

Discover  the 
motivation 
and  meaning 
behind  what 
is  observed 


Data 

Capture  what  is 
observed 


Process 

tracing 

Derive 

patterns 

of  care 

processes 

across 

multiple 

patients 


Field  notes 

Document 
individual 
patient  care 
activities  in 
ICU,  ED 


understanding  guides 
further  observation, 
analysis,  synthesis 


Prototyping 

Create  hard  copy 
and  electronic 
versions  of 
representations 
for  clinicians 
to  use 


Cognitive 
task  analysis 


Account  for  key 
aspects  of  ICU, 
ED  work  domain 


Discover 
how  clinicians 
negotiate 
constraints  to 
achieve  goals 


Audio 
recording 

Document 
verbal 
communication 
in  ICU,  ED 


Phenomena 

Account  for  the 
reality  of  work 
as  it  occurs 


Observation 

Watch,  listen  as  ICU,  ED  clinicians  artifacts 
perform  their  daily  work  routine 


Patient  care 


Communications 


Diagnostic  and 
therapeutic  activity 


Sharing  information 
through  verbal,  visual  means 
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Research  Site 


■  Study  conducted  according  to  USAMRAA  IRB  human  subject 
research  requirements 

■  16  Bed  Burn  ICU  in  tertiary  care  medical  center.  Population 
averages  8  patients,  but  as  high  as  16. 

■  Severe  affliction  from  chemical,  mechanical  or  electrical  burns, 
or  burn-like  afflictions  (e.g.,  toxic  epidermal  necrolysis  (TENS)). 

■  Length  of  stay  from  days  to  months. 

■  Nearby  units  support  BICU,  including  step  down  unit,  burn  OR, 
and  outpatient  clinic. 
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Preliminary  Results 


Clearly  identified: 

■Goals,  and  barriers  to  goals 

■Complex  network  of  relationships  clinicians  maintain  and 
negotiate  to  provide  patient  care. 

■Information  sources  on  which  clinicians  rely 

■Clinician  work  processes,  including  communication 

■Insights  into  work  domain,  operator  issues.  For  example: 
" The  bedside  nurse  is  the  central  figure  of  an  ICU 
patient's  clinician  network." 
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Pop  Quiz! 


Q.  How  many  work  relationships  does  the 
BICU  bedside  nurse  maintain? 
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Preliminary  Results 


Q.  Number  of 
bedside  nurse 
relationships? 

A.  31+ 


BICU 

work 


CRT  Nurse 
Volunteers 
Visitors/Dignitaries 
Nursing  Staff 
Psych 

Family/Friends 

Security 

Administrative  Chief  Nurse 

Radiology 

Physicians 

Bedside  nurse  (for  report) 

Respiratory  therapist 

Occupational  therapist 

Dietician 

Pharmacy 

Researchers 
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Preliminary  Results 

■  The  bedside  nurse  is  informally  responsible  for  a  host  of  other 
activities. 

■  An  important  example  is  reconciling  conflicts  among: 

•  Information  sources 

•  Protocols/guidelines 

•  Unit  policies 

•  Physician  orders 

•  Consultant  recommendations 

•  Care  priorities  among  many  daily  needs 

•  Patient  and  family  preferences  and  requests 
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Example  Data 


Quotes  from  nurse  interviews: 

"Why  can't  the  doctors  make  up  their 
minds  on  what  they  want  me  to  do?" 
(Nurse) 

"Doctor _ [surgeon]  wants  this 

type  of  dressing,  but  the  wound  care 
specialists  want  us  to  use  5%  SMS." 
(Nurse) 

"The  intern  told  me  to  give  the  patient 
PEG,  but  he's  already  on  PO  narcan  and 
we  just  started  feeding  him"  (Nurse) 
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Example  Coded  Data-Theme 


Quotes  from  nurse  interviews:  Barrier 


'"Why  can't  the  doctors  make  up  their 
minds  on  what  they  want  me  to  do?" 
(Nurse) 

"Doctor _ [surgeon]  wants  this 

type  of  dressing,  but  the  wound  care 
specialists  want  us  to  use  5%  SMS." 
(Nurse) 

"The  intern  told  me  to  give  the  patient 
PEG,  but  he's  already  on  PO  narcan  and 
we  just  started  feeding  him."  (Nurse) 


Conflict  between  doctors 


Conflict  between  doctor 
and  Wound  Care  specialist 


Conflict  between  junior 
physician  and  nurse 
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Example  Theme 


Quotes  from  nurse  interviews:  Barrier  Theme 


''Why  can't  the  doctors  make  up  their 
minds  on  what  they  want  me  to  do?" 
(Nurse) 


Conflict  between  doctors 

>  Nurse  reconciles 


"Doctor _ [surgeon]  wants  this 

type  of  dressing,  but  the  wound  care 
specialists  want  us  to  use  5%  SMS." 
(Nurse) 


Conflict  between  doctor 
and  Wound  Care  specialist 

- >  Nurse  reconciles 


"The  intern  told  me  to  give  the  patient  Conflict  between 

PEG,  but  he's  already  on  PO  narcan  and  junior  physician  and  nurse 

we  just  started  feeding  him."  (Nurse)  — ►  Nurse  reconciles 


Page  101  of  132 


22 


W81XWH-12-C-0126 


Implications 

New  understandings  for  the  BICU  staff,  development  team. 

■  The  bedside  nurse: 

•  Is  central  to  ICU  patient  care 

•  Has  unsupported  &  unrecognized  responsibilities  that  detract 
from  direct  patient  care 
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Implications 

■  These  findings  reveal 

•  The  necessary  flexibility  of  the  nurse 

•  Delays  in  decisions  and  care  caused  by  missing  information, 
unresolved  conflicts,  unanswered  questions 

•  The  bedside  nurse  is  the  safety-net  for  the  patient  and  the 
unit. 

■  Computer-based  cognitive  aids  need  to  support  these  aspects 
of  a  nurses  work  in  addition  to  their  direct  patient  care 
activities. 
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Next  Steps 

■  Derive  quantitative  evaluation  criteria  to  compare  clinical 
support  tools 

■  Complete  decision  and  information  requirements 

■  Design,  develop  a  prototype  compatible  with  DoD  IT 
requirements 

■  Test  and  validate  the  prototype  in  concert  with  other  IT 
solutions  that  are  currently  in  use 

■  Field  in  a  clinical  setting 
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We  welcome  your  questions  and  comments 

LTC  Jeremy  Pamplin,  MD 

jeremy.c.pamplin.mil@mail.mil 

Christopher  Nemeth,  PhD 

cnemeth@ara.com 


Our  thanks  to 

Dr.  Shilo  Anders,  Anna  Grome,  Beth  Crandall,  Jeff  Brown, 
Dianne  Hancock,  Greg  Rule,  Nicole  Caldwell 

for  their  instrumental  support  of  this  project,  as  well 
all  of  the  clinicians  who  patiently  participated  in  this  research. 
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True  or  False 

■  The  following  statement  is  taken  from  the  2012  National 
Academies  of  Science  report  on  Healthcare  IT. 

“Medical  and  diagnostic  devices  have  produced  a  therapeutic 
revolution ,  but  in  doing  so  they  have  also  become  more  complex 
and  less  easily  understood  by  those  who  use  them.  When  well 
designed well  made ,  and  properly  used  they  support  and  lengthen 
life.  If  poorly  designed poorly  made ,  and  improperly  used  they 

can  threaten  and  impair  it.  " 
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False 


■  Where  is  it  from? 

•  President  Gerald  Ford,  signing  statement  for  Medical  Device 
Amendments,  May  28,  1976. 


■  We’ve  been  with  this  issue  for  over  45  years,  and  it  is  still 
significant. 
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Emergent  Themes  1 


Theme 

Definition 

Questions 

Rework 

Bridging  and  workaround  strategies  to  link 
systems  that  don’ t  talk  to  each  other. 

Information 

continuity 

ABD  does/doesn’  t  connect  to  Essentris. 
Volume  2  needs  to  be  created  for  long-term 

Pt. 

Transcriptions  FM  WMSNi  to 

Essentris;  Values  too  high  for 

entry. 

Negotiation 

Among  individuals,  specialities,  levels  of 
expertise  dynamic  requiring  negotiation 
hourly/by  shift/daily. 

What?  How  well?  What’ s  fair 
game?  Who?  What  is  off  limits? 

Tacit? 

Scheduling 

Planning,  replanning  among,  across  patients 
and  specialties. 

Anticipation 

Patient  status,  needs  and  how  to  meet  them. 
Preparation  and  participation  in  events. 
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Emergent  Themes  2 

Coordination 

Collaboration  requires  expression  of 
expectations,  prioritization, 
agreement,  recruitment/transfers. 

Who  needs  to  know?  When?  How  far 

ahead?  What  is  information 

appropriate  to  each  role? 

Clarification 

Inquiry,  sensemaking,  common 
grounding  to  reach  threshold  of 
confidence  to  accept  responsibility 
drives  down  level  of  uncertainty. 

Who  do  l/we  know?  Do  l/we  trust  it? 
Implications?  What  else  do  we  need 

to  know?  Do  about  it? 

Resources 

Access.  Availability.  Permission. 
Provision.  Preparation.  Authority. 
Certification.  Use  [“Technical  Work”] 

What' s  available?  Who  controls  it? 

Can  1  get  it  for  my  patient?  When? 
What’ s  difference  between  presumed 
vs.  actual  resources? 

Tasking 

Assignment  of  ICU  staff  to  best  match 
patient  needs.  Individual 
abilities/experience.  Team  needs. 

Who  is  best  match  and  why?  What 

unofficial  information  matters  here? 
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Emergent  Themes  3 


Cross  Check 

Identify/confirm/correct  information 
problem  detection.  Does  it  create 
drag? 

Is  there  a  problem?  Does  this  align 
with  my  expectations?  How  much 
time  does  it  require? 

Tracking 

Account  for  what  needs  to  be  done, 
whether  it  has  been  completed,  what 

remains  to  be  done. 

Who  tracks?  How?  Do  new  items 

modify  or  change  the  set?  Priorities? 

Gaps 

Ability  to  see  “what  isn’ t  there” 

Why  isn’ t  this  here?  What’ s  going  on 
here?  Why  isn’ t  what  l’ m  expecting 
to  be  here  present? 
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Barriers/Challenges 


Theme 

Definition 

Limited  orientation 

Of  residents  and  float  RNs.  RN’ s  fill  gap-takes  time  from  patient 

care 

Lags  in  info:  meds,  labs, 

&  blood 

Rely  on  verbal  orders:  “on  sly”  not  fully  socialized/shared; 
consistent  care  delays 

Bedside  nurse  reconciles 

conflicts 

Technology  protocol,  guidelines,  policy,  regs  vs.  patient  care  needs 
(e.g.  Mixing  ketamine,  consult  note  conflicts) 

Procedural  Drag 

Transcription.  Work  arounds  due  to  system  organizational  gaps 

Reliance  on  memory  as 

failure  marker 

Technology  fails  to  support  needed  work.  E.g.  afternoon  rounds  not 
fed  forward  to  next  day 

Story  of  the  patient/big 
picture  lost 

Trend  info,  understanding  lost/degraded  over  long  term  of  care,  no 
synthesis 

Reliance  on  verbal 

exchanges 

Info  flow  porous,  brittle,  not  shared,  not  reliable;  e.g. 

Acinetobacter  treatment  event;  patient  admission 
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Barriers/Challenges  2 


Authority  gradient 

Encourages  passivity  WRT  concerns;  impediment  to  sharing 

Common  grounding  accuracy 

Under  specification,  confirmation,  verification,  clarification; 
decay  in  treatment,  charge  nurse  may/may  not  address  at  team 

level 

Action/Who  has  the  CON? 

Specialties,  but  no  accountability  on  team  to  assure  results 

Timing 

Lack  of  synchrony,  stal  info;  e.g.  when  procedure  was  performed. 

Salience 

Homogenous  info,  most  relevant  info,  hard  to  find;  stat  orders 

not  evident 

Usability/access/usefulness 

Software  access,  requisite  operator  knowledge,  incorrect  entry 
(e.g.  wound  flow) 

Organizational  issues=drag 

“green  box”  compliance 
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Information:  data  in 
context 


W8 1 XWH- 1 2-C-0 126 


Table 


Sparklines 


Absolute  multigraph 


Relative  multigraph 


Knowledge:  Contextual  information  that  gives 
meaning  (trends,  information  in  context) 


Path  to  achieving 
an  electronic 
system  that  better 
supports  patient 
care? 


Wisdom: 

Applying 
knowledge  to 
new  situations 
and  knowing 
what  is  to  come 
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_ Introduction _ 

•  The  ICU  work  domain  is  complex  and  information  is  dense 

-  Multiple  independent  actors  interacting  in  emergent,  non-linear  relationships 

-  Adaptable  and  resilient 

-  Difficult  to  “unpack"  and  to  understand 

•  Health  IT  has  evolved  to  replace  paper  charting 

-  Vendor-driven 

-  Science  behind  its  structure  and  presentation  is  limited,  at  best 

-  Introduces  potential  safety  risks  and  burdens  to  clinical  work 

•  The  clinical  burden 

-  Information  too  often  missing,  time  delayed,  hidden,  or  erroneous 

-  Multiple,  unconnected  systems,  often  with  multiple  logins  (EHR  documentation, 
radiology,  coding/billing,  inpatient/outpatient,  etc.) 

-  Multiple  screens  of  data,  with  limited/no  salience  (presenting  most  important  data) 

•  What  should  a  display  look  like?  How  should  it  behave? 

•  What  activity(ies)  should  a  computer-based  decision  support  system  assist? 

_ Hypothesis _ 

We  can  describe  the  Burn  ICU  (BICU)  as  a  work  domain  and  account  for  cognitive 
activities  to  identify  design  requirements  for  decision  support  and  communication  tools 
using  Cognitive  Systems  Engineering  (CSE)  methods. 

Objectives 


Preliminary  Results 

•  Preliminary  results  have  identified  the  key  information  sources  that  clinicians  access 
to  make  decisions. 

•  We  broadly  categorized  them  in  five  groups,  as  Figure  1  shows: 

1.  Computer  Based  (e.g.  inpatient  and  outpatient  electronic  health  records,  back  end 
databases)  that  include  information  on: 

•  Laboratory 

•  Radiology 

•  Blood  glucose  management 

•  Wound  care  management 

•  Nutrition  management 

•  Scheduling 

2.  Computer  &  Paper  Based  (e  g.  arterial  blood  gas  results,  printed  vital  signs  from 
monitors) 

3.  Paper  Based  (e  g.  sign-out  sheets,  checklists,  wound  care  plans) 

4.  Other  (e.g.  land  lines  and  cell  phones,  pagers,  text  messages,  e-mail) 

5.  Unclassified  (e.g.  protocols) 

Key  Points 

•  Clinicians  use  numerous  formal  and  informal  information  sources  during  daily 
patient  care. 

•  Computerized  cognitive  aids  intended  to  support  clinician  work  need  to  account 
for  and/or  access  these  information  sources  if  they  are  to  enhance  performance. 


_ Discussion _ 

•  Improving  healthcare  IT,  so  that  it  supports  patient  care  necessitates  going  beyond 
surface  descriptions  (phenotypes)  of  work  domains  to  the  underlying  patterns 
(genotypes)  of  systemic  factors 

•  CSE  methodologies  produce  a  deep  description  of  the  work  domain  studied 

•  Through  observation,  interview,  and  artifact  analysis,  we  demonstrate  that  Burn  ICU 
clinicians  access  numerous  computer,  paper,  computer  and  paper,  and  “other" 
information  systems. 

•  The  design  of  computerized  cognitive  aides  (e.g.  dashboards,  displays, 
communication  systems)  must  account  for  these  information  sources. 

•  Failure  to  account  for  these  information  sources  may  introduce  risk  to  an  IT  system 
causing  potential  patient  safety  concerns  (e.g.  delays,  misses,  lapses,  failures  in  care 
delivery,  etc.),  and/or  clinician  frustration,  workarounds,  and  and  avoidance. 

_ Limitations _ 

•  Single  center/single  unit 

•  Discrete  observation  periods 

_ Conclusion _ 

Understanding  the  sources  of  information  and  who  uses  them  will  facilitate  development 
of  IT  prototypes  that  better  support  clinicians  and  teams  in  their  daily  cognitive  work  to 
improve  their  reliability,  accuracy,  and  efficiency  of  patient  care. 
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To  improve  care  by  supporting  clinical  decision-making  by  identifying  design 
requirements  for  computer-based  decision  support  and  communication  tools. 

_ Methods _ 

•  This  project  is  divided  into  three  phases: 

-  Phase  I:  Foundation  research 

-  Phase  II:  Prototype  development 

-  Phase  III:  Prototype  assessment 

•  Phase  I 

-  Conducted  in  a  16  bed  bum  ICU  in  a  450  bed  tertiary  care,  military,  academic 
medical  center 

-  Five  one-week  data  collection  visits,  each  followed  by  data  analysis  sessions. 

-  Each  visit  includes: 

1)  Direct  observation  of  clinical  teams  providing  patient  care.  Probe  questions 
enable  researchers  to  request  background  and  clarifying  information  situated  in 
context  to  better  understand  motivations,  information  use,  and  decision  making; 

2)  Structured  interviews  elicit  knowledge  from  clinicians  about  their  background, 
perspectives,  work  activity,  information  sources,  and  challenges  they  face; 

3)  Collection  of  computer-based  and  hard  copy  artifacts  that  clinicians  use  in  their 
work.  These  include  sign-out  sheets,  personal  notes,  status  boards,  and 
equipment  displays,  among  others. 

•  Through  data  analysis,  we  develop  descriptive  models  of  the  BICU  work  domain  and 
features  of  clinician  decision-making  and  patient  care.  These  models  describe  the 
content  and  flow  of  information  that  the  project's  prototype  decision  support  and 
communication  system  will  help  to  manage. 


Figure  1.  The  information  sources  clinicians 
use  in  the  Burn  ICU  to  perform  work. 
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expanding  the  realm  of 

POSSIBILITY' 


Research  Topic  Area:  Bio-Informatics  (1) 


Supporting  Warrior  Care 

Findings  from  projects/studies 
aimed  at  promoting,  improving, 
conserving  or  restoring 
personnel  mental  or  physical 
well-being  through  improved 
information  management  &  use 
of  emerging  technologies 
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Objectives  for  This  Session 

■  Become  familiar  with  the  nature  of  and  need  for  human  subject 
field  research  in  military  healthcare  work  settings 

■  Become  familiar  with  the  use  of  human  factors  methods  such  as 
Cognitive  Systems  Engineering  to  understand  and  support  military 
healthcare 

■  Understand  how  human  factors  can  help  to  improve  military 
healthcare  reliability,  safety,  efficiency,  and  resilience. 
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Research  Site 

■  Burn  ICU  in  tertiary  care  medical  center, 

■  16  beds,  2  reserved  to  serve  as  a  post-anesthesia  care  unit  (PACU), 

1  dedicated  to  support  Extracorporeal  Membrane  Oxygenation  (ECMO). 

■  Other  nearby  units  support  the  ICU,  including  a  step  down  unit,  burn 
operating  room,  and  outpatient  clinic. 

■  Population  averages  around  8  patients  but  as  high  as  13 

■  Patients  have  severe  affliction  from  chemical,  mechanical  or  electrical 
burns,  or  burn-like  afflictions  such  as  toxic  epidermal  necrolysis  (TENS). 

■  Length  of  stay  ranges  from  days  to  months. 

I 

I 
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Research  Design 

■  Goal  is  to  improve  care  by  better  supporting  the  judgment  of  individuals 
and  teams  who  care  for  patients  through  a  cognitive  aid  that  also  assists 
communication. 

■  Three  phases  that  are  scheduled  to  take  roughly  a  year  apiece: 
foundation  research,  cognitive  aid  prototype  development,  and 
prototype  assessment. 
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Cognitive  Systems  Engineering  Phase  1 


Preparation 

Understand  the 
domain,  tasks, 
users 

Identify 
cognitively 
complex  tasks 


Knowledge 

Elicitation 

Use  CTA 
methods  to 
understand 
critical  decisions 

Identify  team 
structure  and 
communication 


Analysis  and 

Representation 

•  Decompose  data 
into  discrete 
elements 

•  Identify  user 
decision 
requirements 

•  Identify  the 
central  issues 
and  themes 


\  Application 
Design 

Evaluation 

•  Determine  which 

•  Build  prototype 

metrics  would 

systems  and 

best  measure 

processes 

performance 

•  Transition 

decision 

•  Test  whether 

requirements 

system  supports 

into  design 
concepts 

user 

•  Recommend 

•  Determine  how 

redesigns  to 

to  best  support 

provide  greater 

user  decision 

support 

making  J 

Design  Concepts 

Impact  Estimate 

CCS  Phase  II  and  III 
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BICU  Patient  Team 


Social  Worker 
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Charge  Nurse 

Bedside  nurse  —  Land  line 
Attending  phone 


All  staff  members 


— Cell  phone 


-  Communications 


Attending 


Point  of  care  testing 
Residents/med  students 
Fellow 
Bedside  nurse 
Respiratory  therapist 
Attending 

Bedside  nurse 
Operating  Room  staff 


- Email 

on  phone 


—  Arterial 
blood  gas 


Patient 
-  vital 
signs 
monitor 
printout 


Computer 
and  Paper 


Computer 


Information 

Sources 


Bedside  nurse 
Head  Nurse 
Residents/med  students 


— Protocols 


Charge  nurse  Daily 

Wound  care  team  leader  wound 

care  plan 


Residents/med  students -  Sign  out 

sheet 


Paper 


Charge  Nurse 
Bedside  nurse 
Residents/med  students 


Charge 
—  Nurse 
checklist 
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Patient  health  record - All  staff  members 


Outpatient  health  record 
Lab,  radiology  orders  — 


Residents/med  students 

Fellow 

Attending 

Residents/med  students 

Fellow 

Attending 


Blood  glucose  management—  Bedside  nurse 


Nurse  scheduling 
Radiology  images 


Charge  Nurse 
Nursing  staff 
Residents/med  students 
Fellow 
Attending 


Wound  Flow 


I  All  staff  members  (during  rounds) 
1  Wound  care  nurse  update 


Burn  resuscitation - Bedside  nurse 

decision  support 


Dietary  program - Dietician 

Email - All  staff  members 

Databases  that 
populate  system 
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BICU  Cognitive  Model 


Function  Unit 
Members  Perform 


Synchronization 
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Activities  Unit 
Members  Perform 


Clarification 


Coordination 


Negotiation 


Anticipation 
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Tasks  Unit 
Members  Perform 

Reduce  uncertainty 
Manage  ambiguity 
Manage  conflicting  agendas 
Manage  communication 


Develop  shared  awareness  of  the  Patient 
Get  and  keep  common  ground 
Manage  the  care  plan  and  treatment  goals 
Coordinate  resources 


Identify  alignment,  gaps/differences 
(e.g.,  agendas) 

Manage  conflicting  agendas 
Manage  communication 


Forward  thinking 
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Cognitive  Systems  Engineering  Phases  Two,  Three 


Preparation 

•  Understand  the 
domain,  tasks, 
users 

•  Identify 
cognitively 
complex  tasks 


Knowledge 

Elicitation 

•  UseCTA 
methods  to 
understand 
critical  decisions 

•  Identify  team 
structure  and 
communication 


Analysis  and 

Representation 

•  Decompose  data 
into  discrete 
elements 

•  Identify  user 
decision 
requirements 

•  Identify  the 
central  issues 
and  themes 


CCS  Phase  I 


Application 

Design 

Build  prototype 
systems  and 
processes 

Transition 
decision 
requirements 
into  design 
concepts 

Determine  how 
to  best  support 
user  decision 
making 


Evaluation 

Determine  which 
metrics  would 
best  measure 
performance 

Test  whether 
system  supports 
user 

Recommend 
redesigns  to 
provide  greater 
support 
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Clinician-Specific  View:  RT 
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Unit  View 
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Unit  View:  Schedule 


Page  128  of  132 


12009.  Al  rtgMft 


W81XWH-12-C-0126 


expanding  the  realm  of 

POSSIBILITY' 


Pop  Quiz! 

Q.  What's  the  hardest  part  of  this  project  so  far? 
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Pop  Quiz! 

Q.  What's  the  hardest  part  of  this  project  so  far? 
A.  Access  to  patient  data. 
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Next  Steps 

■  Develop  a  prototype  compatible  with  DoD  IT  requirements 

■Test  and  validate  the  prototype  in  concert  with  other  IT 
solutions  that  are  currently  in  use 

■  Field  in  a  clinical  setting 
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Your  comments  and  correspondence 
are  welcome. 

Christopher  Nemeth,  PhD 
cnemeth@ara.com 
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